Production problems with dataset not updating correctly in Sanity API

Hi,
This is a bit of a shot in the dark, we have some production problems we can't get to the bottom with.

We have a dataset with ~7000 documents, and some of the documents stops working when we add certain data to them.

The problem looks like this:
1. We have a working document
2. Add some data
3. Publish, all looks good in Sanity studio
4. Fetch the document from the API, the changes you made do not appear in the API. The _updateAt date do not change.
5. You can publish how many changes you want and they look like they succeed in the Studio, you get revision history and everything in the Studio. But fetching it from the API returns the version from before the "problematic data" was added.
6. Remove the problematic data, now you get the latest version from the API and everything is fine.


However, the "problematic data" seems to be random.

When we have this unwanted behavior, we discovered that different endpoints gives different data.


Works, we get latest version:


Broken, we get old version until we remove the "problematic data":


(Tested different api versions and it's the same thing)


Anyone experience similar issues that can give some hints?

Could you share some actual query urls with us?

Hi, here is one example:
https://ol1w6ddr.api.sanity.io/v1/data/doc/production/891b828d-fc8c-44a7-bf35-7fbd64d66a3b
https://ol1w6ddr.api.sanity.io/v2021-10-21/data/query/production?query=*[_id%20==%20%22891b828d-fc8c-44a7-bf35-7fbd64d66a3b%22]
The first one gets the latest version:

"_updatedAt":"2022-11-22T14:07:15Z"

The second one gets an old version:

"_updatedAt":"2022-11-15T11:59:04Z"

Here is one more example:
https://ol1w6ddr.api.sanity.io/v1/data/doc/production/e830fa8f-aa23-422f-b668-ee203b8159f0
https://ol1w6ddr.api.sanity.io/v2021-10-21/data/query/production?query=*[_id%20==%20%22e830fa8f-aa23-422f-b668-ee203b8159f0%22]
Response from first endpoint:

"_updatedAt":"2022-11-24T08:14:13Z"

Response from second endpoint:

"_updatedAt":"2022-11-21T07:58:14Z"

Although it's the same document we fetch from the same dataset.

sanity dataset export ...

also returns the correct data, it seems to be just the api that we use to query data that returns old data.

do these things also behave the same in vision (the query playground in the studio)?

Yes, that broken url:
Is copied from vision

We made an export yesterday and restored it to a new dataset, and the new dataset works as expected. But we would like to know what caused it because the export and import is not a sustainable solution since it removes the revision history, and takes time.
The dataset I linked to with this unwanted behavior still exists for debug purposes.

Did you save the other dataset?

Yes, as I wrote above, the dataset with unwanted behavior that I linked to still exists.
The examples still work:

https://ol1w6ddr.api.sanity.io/v1/data/doc/production/e830fa8f-aa23-422f-b668-ee203b8159f0
https://ol1w6ddr.api.sanity.io/v2021-10-21/data/query/production?query=*[_id%20==%20%22e830fa8f-aa23-422f-b668-ee203b8159f0%22]


https://ol1w6ddr.api.sanity.io/v1/data/doc/production/891b828d-fc8c-44a7-bf35-7fbd64d66a3b
https://ol1w6ddr.api.sanity.io/v2021-10-21/data/query/production?query=*[_id%20==%20%22891b828d-fc8c-44a7-bf35-7fbd64d66a3b%22]

Hi,
Did you find anything we did wrong with that dataset?

The new dataset that we imported to still works I think, at least no more cases of this behavior has been reported yet.

Hello sorry for the late reply, we have been super busy with the v3 release and are getting back on track now. I do not have news for you yet, but I am still on it!

Hi Felix - we're checking in with our backend team as it seems like something odd might be going on here indeed. Thank you for providing the details to help us progress this internally. We'll get back to you here on Monday if not earlier.

Great, thank you!

Good morning - we've implemented a potential fix for this in the backend. May we ask you to check if you are getting more consistent results between the two different endpoints now?

Morning,
Yes now it seems to be working! I also see new validation errors as well, which I assumes points out the cause of the issue, which is great!
Thank you much!

That's great, thanks for confirming!