Filtering for unique data in a query and handling duplicate categories in Sanity.io.

7 replies
Last updated: Mar 31, 2021
Hi sanity team Just wanted to know whether is their a way to filter in a query which will result in all unique data.
Eg if we are retrieving title only and there is a title found twice , the filter query should show it only once
Is it possible or is it better we do it at the frontend
AI Update

I need to correct my previous answer - GROQ does not have a built-in unique() function. I apologize for the misinformation.

Unfortunately, GROQ doesn't provide a native way to deduplicate results directly in the query. This means you'll need to handle uniqueness differently depending on your needs:

Since you're retrieving just titles, the most straightforward approach is to deduplicate in your application code:

const titles = await client.fetch(`*[_type == "yourDocumentType"].title`)
const uniqueTitles = [...new Set(titles)]

While this means transferring duplicate data over the network, it's simple, reliable, and performant for most use cases.

Option 2: Prevent Duplicates at the Source

If duplicate titles shouldn't exist in the first place, consider using field validation in your schema to enforce uniqueness:

{
  name: 'title',
  type: 'string',
  validation: Rule => Rule.custom(async (value, context) => {
    const {document, getClient} = context
    const client = getClient({apiVersion: '2023-01-01'})
    const id = document._id.replace(/^drafts\./, '')
    
    const duplicate = await client.fetch(
      `*[_type == $type && title == $title && !(_id in [$id, $draftId])][0]`,
      {
        type: document._type,
        title: value,
        id,
        draftId: `drafts.${id}`
      }
    )
    
    return duplicate ? 'Title must be unique' : true
  })
}

This prevents duplicate titles from being created in the first place.

Option 3: Restructure Your Content Model

If certain titles represent reusable concepts, consider creating a separate document type for them that other documents reference. This naturally enforces uniqueness and creates a single source of truth.

Why No unique() in GROQ?

GROQ is designed as a query language focused on filtering, projecting, and ordering content from the Content Lake. Set operations like deduplication are typically handled at the application layer, giving you more control over which duplicate to keep and how to handle edge cases.

Bottom line: For your specific use case of retrieving unique titles, frontend deduplication is the most practical approach. It's simple and works well. If you need to prevent duplicates altogether, add validation to your schema instead.

It may be possible but this sounds like something that could create some nasty bugs.
Whenever you filter by something, that something you filter by should be unique (unless you are say filtering all documents of a certain type). You can use the sanity isUnique-functionality to enforce uniqness in your slugs.

For things like posts, you can have as many posts with the same title as you want - as long as you treat them by something else in the front end (for example slug, which should always be unique)
My goal is filter out the categories of products (e-commerce), so i can have a groq queries which contains a unique list of categories.
I am filtering all document of a certain type.
Should i move ahead with the sanity function you mentioned
You have duplicate categories in your solution? I usually handle categories as it’s own document that I can pull from, I have not used isUnique for categories - mostly because I usually controll the sites I use and have never inserted duplicated categories.
I would try the funtion, the docs only mention it working for slug but I don’t see why it shouldn’t work for other things as well
What i meant was that there can be more than 1 item with the same category name code-example .I think i should create separate schema for category creation maybe, which will reference to the product schema where we can select the product categories from a dropdown.What do you think?
Hey
user C
just figured out the starter im using has already coded the logic which references the product item. I thought categories were made when we type a category in the product field instead of the dropdown which references the category schema.
Just wanted to tell it’s sorted now , thanks a lot for helping out
I believe they are working on a
destinct
method which will allow to return unique items as per your o.p

Sanity – Build the way you think, not the way your CMS thinks

Sanity is the developer-first content operating system that gives you complete control. Schema-as-code, GROQ queries, and real-time APIs mean no more workarounds or waiting for deployments. Free to start, scale as you grow.

Was this answer helpful?