Migration script not fetching i18n documents with dot notation IDs

18 replies
Last updated: Mar 23, 2023
Hello! How's your Monday going? I'm trying to use the Sanity migration features to accomplish the setting of a new value to every single document. For instance, add a key value:
domain: 'website'

I have used:
https://www.sanity.io/docs/migrating-data
Which works, for document types, images and assets. Hooray! Now, the pickle I'm in, is that I need to migrate
i18n documents as well. HOWEVER, even if:
• I am successfully authenticated
• My role is
administrator
• And my token is
EDITOR
It looks like the, Sanity does not fetch this documents. My query, since I want to pull all documents, is just
*[]

Below I attach a snipped of my migration script. It would be really helpful if anybody had some insight. My current hypothesis is that I can not pull this kind of id:
i18n.<sanity-id>.en_DE
for example because of the restrictions explained here: https://www.sanity.io/docs/ids#fdc25ada5db2
Thank you all in advance
❤️
AI Update

I can help you with this! The issue you're experiencing is related to how Sanity's API handles documents with specific ID patterns, particularly those used by internationalization plugins.

The Core Issue: Document ID Patterns and CDN Access

When using documents with dots in their _id (like i18n.<sanity-id>.en_DE), these are treated as "private" documents by Sanity's CDN. According to the Sanity documentation on document IDs, dots in document IDs create a subpath structure that makes documents private - they won't be exposed through the CDN by default, even with proper authentication.

The Solution: Bypass the CDN

When creating your Sanity client for the migration script, you must set useCdn: false:

import {createClient} from '@sanity/client'

const client = createClient({
  projectId: 'your-project-id',
  dataset: 'your-dataset',
  token: process.env.SANITY_TOKEN,
  useCdn: false,  // Critical for accessing documents with dots in IDs
  apiVersion: '2024-01-01'
})

Setting useCdn: false ensures your queries go directly to Sanity's origin API rather than through the CDN layer. The origin API properly respects your authentication token and will return documents with subpath IDs (like your i18n documents).

Complete Migration Example

Here's how your migration script should look:

import {createClient} from '@sanity/client'
import {defineMigration, at, set} from '@sanity/migrate'

const client = createClient({
  projectId: 'your-project-id',
  dataset: 'your-dataset',
  token: process.env.SANITY_TOKEN,
  useCdn: false,  // Essential!
  apiVersion: '2024-01-01'
})

export default defineMigration({
  title: 'Add domain field to all documents',
  
  async *migrate(documents) {
    // This will now fetch ALL documents, including i18n ones
    for await (const doc of documents()) {
      yield {
        documentId: doc._id,
        patches: [
          at('domain', set('website'))
        ]
      }
    }
  }
})

Understanding Your i18n Setup

Based on your ID pattern i18n.<sanity-id>.en_DE, it looks like you might be using a custom internationalization setup or an older version of an i18n plugin. The document-internationalization plugin uses different patterns:

  • Version 2+: Uses translation.metadata documents
  • Version 1: Uses IDs like {base-id}__i18n_{locale}

Your pattern with dots is creating private documents, which is why the CDN isn't returning them.

Additional Troubleshooting

  1. Test your query first: In Vision plugin, verify the documents exist:

    *[_id match "i18n.*"]
  2. Verify token permissions: Your EDITOR token should have read/write access to all document types.

  3. Check the migration client: Make sure the client you pass to the migration functions also has useCdn: false set.

The useCdn: false setting is the key to solving your issue - it's a common gotcha when working with documents that have special ID patterns!

Show original thread
18 replies
That’s odd. That restriction only applies to unauthenticated users.
Is it possible that your token is not being picked up? What command are you using to execute the script?
Hi!
user M
I hope not 😂 I am using the following command:
sanity exec src/scripts/migrate.ts --with-user-token
Can you try running
sanity logout & sanity login
? Make sure that you’re using the same method to login as you did when you created the account.
Yes, but that did not work. If I don't do it in batches of 100, and I request all documents, then it will signal the problem.


/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/@sanity/client/src/http/request.ts:13
      throw new ClientError(res)
            ^

Error: Mutation(s) failed with 1 error(s):
- Insufficient permissions; permission "manage" required
    at res (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/@sanity/client/src/http/request.ts:13:13)
    at applyMiddleware (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/get-it/src/util/middlewareReducer.ts:8:15)
    at onResponse (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/get-it/src/createRequester.ts:91:22)
    at cb (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/get-it/src/createRequester.ts:58:65)
    at callback (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/get-it/src/request/node-request.ts:52:57)
    at cb (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/get-it/src/request/node-request.ts:145:14)
    at DestroyableTransform.<anonymous> (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/get-it/src/request/node/simpleConcat.ts:8:13)
    at Object.onceWrapper (events.js:519:28)
    at DestroyableTransform.emit (events.js:412:35)
    at endReadableNT (/Users/pol.ruiz/dev/wefox-web-website-cms/node_modules/readable-stream/lib/_stream_readable.js:1010:12) {
  statusCode: 403,
  response: {
    body: { error: [Object] },
    url: '<https://xwqbll4b.api.sanity.io/v2023-03-01/data/mutate/development?returnIds=true&visibility=sync>',
    method: 'POST',
    headers: {
      'content-type': 'application/json; charset=utf-8',
      'content-length': '253',
      'x-ratelimit-remaining-second': '49',
      'x-ratelimit-limit-second': '50',
      'ratelimit-limit': '50',
      'ratelimit-remaining': '49',
      'ratelimit-reset': '1',
      date: 'Tue, 21 Mar 2023 12:16:48 GMT',
      'server-timing': 'api;dur=2201',
      'x-sanity-shard': 'gcp-eu-w1-01-prod-1042',
      'x-served-by': 'gradient-web-67ddfd8698-jgv5r',
      'strict-transport-security': 'max-age=15724800; includeSubDomains',
      vary: 'origin',
      xkey: 'project-xwqbll4b, project-xwqbll4b-development',
      via: '1.1 google',
      'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
      connection: 'close'
    },
    statusCode: 403,
    statusMessage: 'Forbidden'
  },
  responseBody: '{\n' +
    '  "error": {\n' +
    '    "description": "Mutation(s) failed with 1 error(s)",\n' +
    '    "items": [\n' +
    '      {\n' +
    '        "error": {\n' +
    '          "description": "Insufficient permissions; permission \\"manage\\" required",\n' +
    '          "permission": "manage",\n' +
    '          "type": "insufficientPermissionsError"\n' +
    '        },\n' +
    '        "index": 197\n' +
    '      }\n' +
    '    ],\n' +
    '    "type": "mutationError"\n' +
    '  }\n' +
    '}',
  details: {
    description: 'Mutation(s) failed with 1 error(s)',
    items: [ [Object] ],
    type: 'mutationError'
  }
}
We have an advanced plan. But in theory that should be enough to make migrations right?
My token is set as
EDITOR
as well and my role is
administrator
. What's the missing piece here
user M
Migrations are available on all plans. Given the error, I still think you’ve logged in using a different provider than you did on the project. The editor token you’re passing does not matter since you’re using the the
--with-user-token
flag.
Perhaps this is something related to your problem
user B
https://sanity-io-land.slack.com/archives/C9Z7RC3V1/p1679405448464339
Gist is:

The problem was that i got some v2 code in there and that is why it was not running.
(Use of 'useClient' instead of 'createClient' as in the example)
Hi folks! For closure In the end I found the issue. 🌈 The problem was that upon using a query like
*[]
. I was pulling all documents. In my project, this was a total of nearly 914 documents. After creating a JSON with the document
_id
and
_type
, I saw, that all documents were pulled correctly including the ones with
.
and
i18n
. However, this wasn't the only thing this query was pulling. At the beginning I thought I was pulling only:
• Custom types that we created.
• Image assets types
• File assets types
However, there were also, types which included:

system

sanity
In this case, the error was that this elements weren't filtered out, as the query was requesting every single document. Upon attempting to modify documents with this type, I got the error regarding permissions, which would make sense as I believe
document types with this extension might be protected. After properly filtering this types, I saw that I was getting all documents without restrictions since the authentication was correct and the token was correct 🎉
Let’s use greetings like “Hey Everyone,” “Hi, Y’all," or “Hello, Folks” to make sure everyone in the community is included. Please read the Sanity Community Code of Conduct to stay updated on expected communication &amp; behavior in our spaces: https://www.sanity.io/docs/community-code-of-conduct
user G
user M
Thanks for reassuring some things and for your support 🙌
Glad you got it sorted out! I hadn’t considered that system documents would be causing it!
That's okay! Perhaps you could recommend this addition as a Gotcha, inside this documentation page: https://www.sanity.io/docs/migrating-data
I couldn't find a warning about this inside the docs 🤗
Great idea!
welcome
user B
. so glad you’ve got this sorted now. 😊

Sanity – Build the way you think, not the way your CMS thinks

Sanity is the developer-first content operating system that gives you complete control. Schema-as-code, GROQ queries, and real-time APIs mean no more workarounds or waiting for deployments. Free to start, scale as you grow.

Was this answer helpful?