Imported documents not showing without token, but manually created ones work

Based on your description, this is almost certainly a perspective issue related to how your imported documents were created. Here's what's likely happening:

When you import documents via the Sanity CLI, they're typically created as draft documents with IDs prefixed by drafts. (like drafts.post-123). Even though these documents appear as "published" in the Studio, they may not have corresponding published versions (without the drafts. prefix) in your Content Lake.

Meanwhile, when you manually create documents in the Studio and publish them, Sanity properly creates both the draft version AND the published version, which is why those show up on your site.

Why This Matters

As of API version 2025-02-19, the default perspective changed from raw to published. This means:

• Without a token (production): Your API queries use the published perspective by default, which only returns documents without the drafts. prefix
• With a token (authenticated): You can access different perspectives including raw (all documents) or previewDrafts (drafts + published)

How to Fix This

You need to "publish" your imported documents so they exist as published versions. Here are your options:

Option 1: Bulk publish via CLI script

Create a script that publishes all your imported documents:

import {createClient} from '@sanity/client'

const client = createClient({
  projectId: 'your-project-id',
  dataset: 'your-dataset',
  token: 'your-token-with-write-access',
  apiVersion: '2025-02-19',
  useCdn: false
})

// Fetch all draft documents
client.fetch('*[_id in path("drafts.**")]').then(drafts => {
  const transaction = client.transaction()
  
  drafts.forEach(draft => {
    const publishedId = draft._id.replace('drafts.', '')
    const publishedDoc = {...draft, _id: publishedId}
    delete publishedDoc._rev // Remove revision to avoid conflicts
    transaction.createOrReplace(publishedDoc)
  })
  
  return transaction.commit()
})

Option 2: Modify your import script

When importing, create both draft AND published versions:

// For each document you import:
const publishedDoc = {_id: 'my-doc-id', ...data}
const draftDoc = {_id: 'drafts.my-doc-id', ...data}

await client.createOrReplace(publishedDoc)
await client.createOrReplace(draftDoc)

Option 3: Use the Studio UI

If you have a manageable subset, you can open each document in the Studio and click "Publish" to create the published version.

Verify the Fix

After publishing, verify by querying without a token:

// This should now return your documents
fetch('https://your-project.api.sanity.io/v2025-02-19/data/query/production?query=*[_type=="yourType"]')

The key insight is that "published" in Sanity means the document exists without the drafts. prefix, not just that it has a published status flag. Your import process likely only created the draft versions, which is why they only show up when you use a token (which gives you access to the raw or previewDrafts perspectives).