We have about 13,000 documents - most of which we have imported manually using the sanity cli. For some reason none of these are showing up on our site without using a token....

We have about 13,000 documents - most of which we have imported manually using the sanity cli.
For some reason none of these are showing up on our site without using a token. What's odd is that when I manually create a new document, it shows up just fine. I
have checked several imported documents to see they meet the required fields and are published - but to no avail.
I have no idea why imported documents are not showing, but newly created ones are.

Unfortunately we're realising this just a few days before the content team are taking over to update content before a big launch.

• The dataset is public• I've made sure the documents meet validation requirements
• I've made sure documents are published (even toggled on and off)
• They DO show when using a token so I don't think it has anything to do with my groq queries

When you export your dataset, can you see any differences between the documents you originally imported and the new ones you created manually?
Also, if you reimport, do the documents you had created manually no longer appear?

Are there any dots

.

in the imported document ids? Documents on a path will always be private https://www.sanity.io/docs/ids#paths-fdc25ada5db2

Ah yes a lot of them (not all) have dots, like
post.some-id-here
author.some-id-here

This is because I read somewhere in the Sanity docs that using this will allow for some more interesting grow queries.

Considering how many documents we have, can you think of an affordable solution to this?
Changing each one will drive up our quota a lot.

Yes, it's a bit of a hidden footgun that paths are always private. Sorry about that.
Running a script that strips off that path in the ids should be fairly straight forward. You can do multiple (hundreds) of mutations in a single transactions. I'll see if I can dig out an example for you..

I think the "easiest" solution for you would be to use the CLI to1. export the dataset

sanity dataset export

2. modify the data locally (strip off the paths) using a script or search and replace
3. import the dataset

sanity dataset import

to a new name4. update the studio and front-end to use this new dataset

(since document ids are immutable using a script to mutate would be a bit more involved)

Thank you! Just to clarify though, as they would have different id's - reimporting then wouldn't replace the previous document. That's why I need a new dataset?

Yes, that's why I thought having a new dataset would be the best solution. Otherwise you would have to delete all the "old" documents.

Also, I just realised.. are there any references to the documents on paths? If so you will have to update all the references as well with the updated ids

Fortunately not many of them are referenced, and those that are will be caught by the editors.
Thanks a lot