How are you doing A/B experiments in Sanity?

15 replies

Last updated: Jan 11, 2023

user T

hey great demo today. I might have missed it but is there any info on how you're doing A/B experiments anywhere?

AI Update

Great question! Sanity has solid support for A/B testing through the @sanity/personalization-plugin. Here's what you need to know:

The Plugin

The personalization plugin lets you run A/B (or A/B/N) experiments at the field level in your content. Instead of duplicating entire documents or pages, you can create variants for specific fields like headlines, button text, descriptions, or any other content field you want to test.

How It Works

You configure which field types should support experiments in your sanity.config.ts:

import {fieldLevelExperiments} from '@sanity/personalization-plugin'

export default defineConfig({
  plugins: [
    fieldLevelExperiments({
      fields: ['string', 'text'], // field types to experiment on
      experiments: [
        {
          id: 'hero-headline',
          label: 'Hero Headline Test',
          variants: [
            { id: 'control', label: 'Control' },
            { id: 'variant-a', label: 'Variant A' },
          ],
        },
      ],
    })
  ],
})

This creates new field types (like experimentString) that your editors can use in Studio to manage multiple content variants.

Integration with External Tools

The plugin works great with external experimentation platforms like LaunchDarkly, GrowthBook, and Amplitude. You can either hardcode your experiments in the config or use an async function to fetch them dynamically from your experimentation service.

A real-world example: Zego (UK insurance company) uses this setup with LaunchDarkly—their marketing team creates variants in Sanity while controlling traffic distribution through LaunchDarkly.

Querying Variants

You can filter variants either server-side using GROQ's coalesce function or client-side in JavaScript. The server-side approach looks like:

*[_type == "page"][0]{
  "headline": coalesce(
    headline.variants[experimentId == $experiment && variantId == $variant][0].value,
    headline.default
  )
}

Learning Resources

Sanity has a full A/B Testing course on Sanity Learn that walks through setup, implementation with Next.js, and integration with external services.

The approach is pretty flexible—you control content variants in Sanity while your experimentation platform handles user bucketing and analytics. Companies like Eurostar have reported that CMS tasks that used to take 2-3 weeks now take 2-3 days with this setup!

Show original thread

15 replies

In that demo I have an object called “experiment”, which contains an array for two references.
In my GROQ query I only resolve one of them, based on an

$audience

variable which is set to either 0 or 1.
In the live preview I also had a way to change the audience variable so you could at least preview both.

It’s a simple example in that it doesn’t then do the sort of tracking you would want to analyze the “score” of which one did better. But that’s where you’d likely need to integrate your own solution.

Ah okay, what I was hoping to see was some way to manage who sees which experiment

Maybe that could be some randomized number

Math.round(Math.random())

Will give you a

for the first load
But you’d probably also want to write that to a cookie/localStorage so that you’re consistently serving the same one each time.

Might be worth tying to some vercel edge function so we have location based testing too

Yep for sure! I’ve got a personalisation demo that feels the Vercel Geo headers into a variable as well.

Oh sweet is it shared?

The API route looks like this:
From memory though,

x-vercel-ip-country

is only present for Enterprise Vercel accounts

import {getClient} from '../../lib/sanity.server'

export default async function getBanner(req, res) {
  const country = req?.query?.country || req?.headers?.['x-vercel-ip-country']

  if (!country) {
    return res
      .status(400)
      .json({message: `No Country in Params or Request Headers`, headers: req.headers})
  }

  const data = await getClient().fetch(
    `*[_type == "banner" && upper(country) == upper($country)][0]`,
    {country}
  )

  if (!data) {
    return res.status(404).json({message: `No Banner found for Country`, country})
  }

  return res.status(200).json(data)
}

Thanks a million I'll look into this

In that demo I have an object called “experiment”, which contains an array for two references.

user T

is the "experiment" object inside the "Home" document? Can you share how you did this?

The “Home” document is a just a singleton “Page”.
The experiment object schema looks like this:

import {Wand2} from 'lucide-react'
import {defineField} from 'sanity'

export default defineField({
  name: 'pageBuilderExperimentCell',
  title: 'Experiment',
  type: 'object',
  icon: Wand2,
  fields: [
    defineField({
      name: 'experiments',
      description: 'Choose 1-2 blocks to be split between A/B audiences',
      type: 'array',
      of: [
        defineField({
          name: 'experiment',
          type: 'reference',
          to: [{type: 'article'}],
        }),
      ],
      validation: (rule) => rule.max(2).required(),
    }),
  ],
  preview: {
    select: {
      title0: 'experiments.0.title',
      title1: 'experiments.1.title',
    },
    prepare: ({title0, title1}) => ({
      title:
        title0 || title1
          ? [title0, title1]
              .filter(Boolean)
              .map((title) => `"${title}"`)
              .join(` vs `)
          : `No "Articles" selected`,
      subtitle: 'Experiment',
      media: Wand2,
    }),
  },
})

and in my GROQ query I only resolve one of these, based on a variable passed into the query:

experiments[$audience]

$audience is either

Cool, I like this idea, thank you so much for sharing the code!
Where do you add this object? Do you create a document like "Experiments" and add this object to it, or do you add it to the "Articles" document?

In this particular demo, it’s in the Page Builder array of a Page document.

Oh, I got it! This is good to test parts of the page. In my case I need to run an experiment with two pages, so I think the solution is to create a document "A/B Tests" where I can add the "experiments" object which contains references for "Pages" document. And I can use the query you shared to get the right experiment based on

audience

Good idea!

Sanity – Build the way you think, not the way your CMS thinks

Sanity is the developer-first content operating system that gives you complete control. Schema-as-code, GROQ queries, and real-time APIs mean no more workarounds or waiting for deployments. Free to start, scale as you grow.

Get started for free Explore the demo