# Fetch your collection items from the API

https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/fetch-collection-items-api

Paginate through every Webflow collection and write items to disk as NDJSON, one record per line, so large collections never accumulate in memory.

---

## Navigation

**Course:** [Migrating from Webflow to Sanity](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity) · [View as markdown](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity.md)

**Previous:** [Handle internal links in Portable Text](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/internal-links-portable-text) · **Next:** [Run the asset pre-upload pass](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/asset-pre-upload)

---

## Course Contents

1. [Introduction](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/introduction-to-webflow-migration)
2. [Audit your Webflow content](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/audit-webflow-content)
3. [Recognize Webflow's content modeling patterns](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/recognize-content-modeling-patterns)
4. [Export from Webflow and connect the Data API](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/export-webflow-connect-api)
5. [Initialize Sanity and install the migration toolkit](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/initialize-sanity-migration-toolkit)
6. [Map collections to Sanity document types](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/map-collections-to-schemas)
7. [Build a page builder with a sections array](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/page-builder-sections-array)
8. [Model custom-template pages as singletons](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/custom-template-singletons)
9. [Model navigation in Sanity](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/model-navigation)
10. [Create a siteSettings singleton](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/site-settings-singleton)
11. [Handle internal links in Portable Text](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/internal-links-portable-text)
12. **Fetch your collection items from the API** *(current)*
13. [Run the asset pre-upload pass](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/asset-pre-upload)
14. [Resolve reference fields from API data](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/resolve-reference-fields)
15. [Convert rich text to Portable Text](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/convert-rich-text-portable-text)
16. [Extract static page content using the page DOM API](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/extract-static-page-content)
17. [Import content with the two-pass pattern](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/two-pass-import)
18. [Validate your import in Studio](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/validate-import-studio)
19. [Set up forms, redirects, and sitemap](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/forms-redirects-sitemap)
20. [Set up draft mode and visual preview](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/draft-mode-visual-preview)
21. [Validate your import without a frontend](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/validate-migration)
22. [Launch checklist and DNS transfer](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/launch-checklist-dns)
23. [AI prompt library and resources](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/appendix-prompt-library-resources)

---

With your schema finalized and Studio running locally, fetch your collection items. The script below writes items to disk line-by-line as each page arrives. It doesn't accumulate the full collection in memory before saving. This works the same whether your collection has 500 items or 100,000.



Items are saved as NDJSON (newline-delimited JSON): one JSON object per line, no wrapping array. Your import scripts read and stream these files directly, which is how @sanity/import expects them.



### Write the streaming fetcher



```javascript:scripts/fetch-collection-items.js
// scripts/fetch-collection-items.js
import fs from 'fs'
import { createWriteStream } from 'fs'

const TOKEN = process.env.WEBFLOW_API_TOKEN
const BASE_URL = 'https://api.webflow.com/v2'
const HEADERS = { Authorization: `Bearer ${TOKEN}`, Accept: 'application/json' }

// Back off on 429 responses using the retry-after header
async function fetchWithBackoff(url, attempt = 0) {
  const res = await fetch(url, { headers: HEADERS })
  if (res.status === 429) {
    const wait = parseInt(res.headers.get('retry-after') ?? '10', 10) * 1000
    const delay = wait * Math.pow(2, attempt)
    console.log(`  Rate limited — waiting ${delay / 1000}s`)
    await new Promise(r => setTimeout(r, delay))
    return fetchWithBackoff(url, attempt + 1)
  }
  if (!res.ok) throw new Error(`${res.status} ${res.statusText} — ${url}`)
  return res.json()
}

// Fetch one collection and write items to NDJSON, one line per item
async function fetchCollectionToNDJSON(collectionId, outputPath) {
  const stream = createWriteStream(outputPath)
  let offset = 0
  const limit = 100
  let total = null
  let written = 0

  while (true) {
    const data = await fetchWithBackoff(
      `${BASE_URL}/collections/${collectionId}/items?limit=${limit}&offset=${offset}`
    )
    if (total === null) total = data.pagination.total

    for (const item of data.items) {
      // Skip drafts and archived items here — keeps output files clean
      if (item.isDraft || item.isArchived) continue
      stream.write(JSON.stringify(item) + '\n')
      written++
    }

    process.stdout.write(`\r  ${Math.min(offset + limit, total)} / ${total}`)
    offset += limit
    if (offset >= total) break
  }

  await new Promise(resolve => stream.end(resolve))
  process.stdout.write('\n')
  return written
}

async function run() {
  const { collections } = JSON.parse(
    fs.readFileSync('webflow/api/collections.json', 'utf8')
  )
  fs.mkdirSync('webflow/api/items', { recursive: true })

  for (const col of collections) {
    const outputPath = `webflow/api/items/${col.id}.ndjson`
    console.log(`Fetching: ${col.displayName}`)
    const count = await fetchCollectionToNDJSON(col.id, outputPath)
    console.log(`Saved ${count} items → ${outputPath}`)
  }
}

run().catch(err => { console.error(err.message); process.exit(1) })
```

Run it from the project root:



```bash
node --env-file=.env scripts/fetch-collection-items.js
```

Each line in the output file has this shape:



```json
{"id":"64e7f4a8b0e1234567890abc","isDraft":false,"isArchived":false,"fieldData":{"name":"My Post Title","slug":"my-post-title","author":"64e7f4a8b0e1234567890def","tags":["64e7f4a8b0e111","64e7f4a8b0e222"],"body":"<p>Rich text as HTML...</p>"}}
```

`fieldData` contains all CMS field values using your collection's field slugs. Reference fields — single and multi — are Webflow item ID strings. That direct ID-to-ID mapping is what makes API-based migration cleaner than CSV exports.



**Scale note: **Enterprise Webflow plans support 20,000 to 100,000+ items per collection. The NDJSON streaming approach above handles any size. Memory usage stays flat because items are written to disk as each page arrives rather than accumulated first. The rate-limit backoff handles the roughly 1,000 API calls needed to fetch a 100k collection.



The bottleneck at this scale shifts to the import side. Sequential `client.create()` calls at 100k items would take hours. For collections over roughly 1,000 items, use `@sanity/import` with a pre-assigned ID strategy instead. See [the import lesson](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity/two-pass-import) for the streaming approach.



Next, you’ll upload all Webflow CDN assets to Sanity before creating any documents.



---

## Related Resources

- [Full course as markdown](https://www.sanity.io/learn/course/migrating-content-from-webflow-to-sanity.md)
- [All courses and lessons](https://www.sanity.io/learn/sitemap.md)
- [Complete content for LLMs](https://www.sanity.io/learn/llms-full.txt)

<AgentInstructions>

## Submitting Feedback

If you encounter incorrect, outdated, or confusing documentation on this page, submit feedback:

POST https://www.sanity.io/learn/resource/feedback

```json
{
  "path": "/learn/course/migrating-content-from-webflow-to-sanity/fetch-collection-items-api.md",
  "feedback": "Description of the issue"
}
```

Only submit feedback when you have something specific and actionable to report.

</AgentInstructions>