# Validating incoming content

https://www.sanity.io/learn/course/refactoring-content/validating-incoming-content.md

Never trust your existing content source. Validate all data during a migration to avoid future headaches.

The formatting of your existing content may be problematic or insufficient. Examples include:

* You may need to escape HTML entities or process Markdown formatting.
* You may want to trim strings to remove whitespace.
* Integers may need to be converted to strings – or vice-versa.

## Use TypeScript

TypeScript lets you add data type definitions to JavaScript. It's beneficial for migration projects because you probably want to go from messy and idiosyncratic content (we assume!) to tidy and structured content.

### Sanity TypeGen

If you have started configuring your content model for Sanity Studio, you can use Sanity TypeGen to generate types for it and use those types for the output in your migration scripts.

1. Check out the [Generating types](https://www.sanity.io/learn/course/day-one-with-sanity-studio/generating-types) lesson.

Example of using generated types in a migration script:

```typescript
import { Post } from './sanity.types' // From Sanity TypeGen

export default defineMigration({
  title: 'Import WP JSON data',

  async *migrate(documents) {
    const wpType = "posts";
    let page = 1;
    let hasMore = true;

    while (hasMore) {
      try {
        const wpData = await wpDataTypeFetch(wpType, page);

        if (Array.isArray(wpData) && wpData.length) {
          for (const wpDoc of wpData) {
            const doc: Post = {
              _id: `post-${wpDoc.id}`,
              _type: "post",
              // Add other required fields here based on wpDoc structure
            };

            yield createOrReplace(doc);
          }
          page++;
        } else {
          hasMore = false;
        }
      } catch (error) {
        console.error(`Error fetching data for page ${page}:`, error);
        hasMore = false; // Stop the loop in case of an error
      }
    }
  },
});
```

### Runtime validation

You can also consider a runtime validation library such as Zod to validate, catch, transform or throw errors for any unexpected problems with incoming data.

1. [Zod is a popular library](https://zod.dev/) for validating content at run time.

## Validate post-migration data

Your Sanity Studio schema types should also include validation rules on all content. This way, as your content is migrated into the Content Lake, you can validate all new documents against these rules from the terminal by running:

```sh
npx sanity@latest documents validate
```

1. See the documentation about [Validation](https://www.sanity.io/learn/studio/validation) on schema types and [validating all documents with the CLI](https://www.sanity.io/docs/documents#0ee72ece4609).