Error importing data into Sanity via CLI due to malformed data and internal server error.

I understand you're hitting an "Internal server error" when trying to import an NDJSON file into Sanity. This is a frustrating error because it's not immediately clear what's causing it. Let me help you troubleshoot this.

The "Internal server error" during sanity dataset import typically indicates one of these issues:

Common Causes and Solutions

1. Document Size Limits

Sanity has document size limits. If your sources.ndjson file contains any documents that are too large, the import will fail with an internal server error. According to the importing data documentation, check your NDJSON file for unusually large documents:

# Check individual document sizes (line length as rough proxy)
awk '{print length, NR}' sources.ndjson | sort -rn | head

If you find very large documents, you'll need to split them or reduce their size before importing.

2. Invalid NDJSON Format

Each line must be a valid, complete JSON object. Common issues include:

• Missing commas or brackets
• Invalid escape characters
• Non-UTF8 characters
• Empty lines

Validate your NDJSON format:

# Test if each line is valid JSON
while IFS= read -r line; do echo "$line" | jq empty; done < sources.ndjson

3. Schema Validation Issues

If your documents don't match your current schema (missing required fields, invalid field types), the import can fail. Make sure your schema is deployed and matches the data structure in your NDJSON file.

4. Asset References

If your NDJSON references assets that don't exist in the target dataset, this can cause issues. The import documentation notes that assets need to be available when importing documents that reference them.

Debugging Steps

Enable debug logging to get more detailed error information:

# Set debug logging level
SANITY_LOG_LEVEL=debug sanity dataset import ../json/ndjson/sources.ndjson production --replace

Or use the --debug flag if available in your CLI version.

Try Importing in Smaller Batches

If the file is large, try splitting it into smaller chunks:

# Split into files with 100 documents each
split -l 100 sources.ndjson sources-part-

# Import each part
for file in sources-part-*; do
  sanity dataset import "$file" production
done

This helps identify which specific document(s) are causing the issue.

Alternative: Use the Client Library

If CLI import continues to fail, you can use the JavaScript client to import documents with better error handling:

import {createClient} from '@sanity/client'
import fs from 'fs'

const client = createClient({
  projectId: 'your-project-id',
  dataset: 'production',
  token: 'your-token',
  useCdn: false
})

const ndjson = fs.readFileSync('sources.ndjson', 'utf8')
const documents = ndjson.split('\n').filter(Boolean).map(JSON.parse)

for (const doc of documents) {
  try {
    await client.createOrReplace(doc)
    console.log(`Imported: ${doc._id}`)
  } catch (err) {
    console.error(`Failed to import ${doc._id}:`, err.message)
  }
}

This gives you line-by-line error reporting to identify problematic documents.

Check API Rate Limits

As mentioned in the importing data guide, if you're doing a programmatic import, make sure you're respecting API rate limits by using a queue with reasonable concurrency.

If none of these solutions work, I'd recommend reaching out to Sanity support with the debug logs and a sample of your NDJSON file (with sensitive data removed) so they can investigate the specific internal server error.