Golden dataset definition

A golden dataset in AI evaluation is the trusted set of example inputs, each with a human-agreed correct output or grading rubric, that you score a model against before shipping a change. Its value depends on the inputs staying identical between runs. In Sanity, every write to a document produces a new revision ID, so an evaluation example can store a document's `_id` and `_rev` and re-fetch the exact text its label was written against, even after editors have rewritten the live version.

What is a golden dataset in AI evaluation?

A golden dataset is the collection of test examples an AI team trusts enough to gate a release: real inputs, paired with the output a reviewer has confirmed is correct, kept separate from anything the system was trained or tuned on. It goes by several names depending on the team and the era: gold set, gold standard set, test set, eval set, reference set, or ground truth set. In classical machine learning the same artifact was simply the test set.

The pairing is looser than people expect. For a classification task, the known-correct output is a label, and scoring is a string or category match. For open-ended generation there is often no single correct string, so the "golden" side of the pair is a rubric, a list of facts the answer must contain, or a pass/fail criterion a human or a calibrated judge applies. Both count as a golden dataset. What makes it golden is not the format of the answer but the fact that a person reviewed it and the team agreed to be held to it.

A golden dataset is also not the same thing as an eval. The eval is the whole apparatus: the dataset, plus the graders, plus the metrics, plus the harness that runs them, plus the threshold that decides pass or fail. The golden dataset is the data component, and it is usually the component that determines whether the rest of the apparatus tells you anything true.

How do you build a golden dataset?

Building a golden dataset is a sampling and labeling exercise, not a writing exercise. The sequence that holds up in practice runs roughly like this.

First, sample from real traffic rather than inventing examples. Production logs, support tickets, and sessions where a user corrected the system are the richest seams, because they show what the system is actually asked and where it actually breaks. Second, stratify the sample so that high-volume, easy cases do not crowd out everything else. Useful axes are intent, channel, content type, customer segment, and difficulty. Third, add edge cases, adversarial inputs, and every past failure you can recover from incident notes, deliberately, because random sampling will not surface them at a useful rate. Fourth, label and review with people: one labeler produces a draft, and a second reviewer is what makes it golden. Fifth, record metadata on every example, including slice tags, where the input came from, who labeled it, when, and what evidence supports the label. Sixth, version the whole set, and treat a change to it as an event you can diff rather than a quiet edit.

Sizing is a trade-off you should make on purpose. The set has to be large enough that the regression you care about is bigger than the sampling noise, so the smaller the regression you want to detect, the more examples you need, and every example costs human review time. That is why many teams keep tiers: a small set that runs on every commit to catch gross breakage, and a larger one that runs before a release.

How do you keep a golden dataset honest?

A golden dataset stays honest through four disciplines: do not tune against it, version it, refresh it, and keep its inputs reproducible.

Do not tune against it. If you iterate on prompts, retrieval settings, or model choice while watching the golden set score, the set has quietly become a development set, and the number stops predicting production behavior. Keep a separate iteration set for that work and look at the golden set rarely. Even without any data leakage, repeatedly selecting changes by their score on one fixed set fits the system to that set. Recht and colleagues demonstrated the effect by building entirely fresh test sets for a long-reused benchmark in Do ImageNet Classifiers Generalize to ImageNet? at ICML 2019.

Version it alongside the system. Two scores are only comparable if you can prove the set was identical, which means the dataset needs the same change history discipline as code. Refresh it as the domain changes, because a golden dataset built on last year's product catalog measures last year's product catalog.

Keep the inputs reproducible. When the inputs are content you own in a repository, pin each example to a specific document revision rather than copying the text into the eval record and hoping it matches. Sanity is the AI Content Operating System, and the Content Lake gives every document a revision ID that changes on every write, with a history endpoint that returns the document as it was at that revision. One honest caveat: revision history is retained for a plan-dependent window (3 days on Free, 90 days on Growth, 365 days or custom on Enterprise), after which older revisions are truncated. So for a long-lived set, snapshot the resolved JSON into the eval record and keep the revision ID as a provenance pointer.

What goes wrong with golden datasets?

The failure modes of a golden dataset are well documented, and most of them end with a score that looks fine while the system gets worse.

Contamination, also called leakage, is the first. The evaluation examples, or near-duplicates of them, are present in the model's training data, so the score measures memorization rather than capability. This is measured, not hypothetical: see the survey of benchmark data contamination and the open-source contamination report published in Findings of EMNLP 2024. It is also the structural reason a private golden dataset beats a public benchmark for your own release decisions. A public benchmark is standardized and comparable across vendors, which is exactly why it ends up in training corpora and why it does not reflect your traffic.

Label disagreement is the second. Two competent annotators give different answers and neither is wrong. Barbara Plank's The "Problem" of Human Label Variation argues that this variation is signal rather than noise, and that the assumption of a single ground truth is often the thing that is broken. The practical consequence is that an example your labelers cannot agree on is telling you the task definition is underspecified. Fix the rubric, not the labeler.

Only easy cases is the third. A set sampled uniformly from traffic is mostly happy path, will score in the nineties, and will tell you nothing about the inputs that cause incidents.

Stale inputs is the fourth. When the input is a document other people can edit, the label silently decays and nobody notices until the eval contradicts reality.

Mistaking consensus for truth is the fifth. Scoring agreement with other models, or with an LLM-as-judge, measures agreement, not correctness. A judge has to be calibrated against human-labeled examples first, which means a judge-based setup needs a golden dataset more than a string-match setup does, not less.

Why does structured content make golden datasets easier to maintain?

Structured content makes a golden dataset easier to maintain because it gives both halves of each example a stable shape: a bounded input you can re-fetch exactly, and an answer space defined by a schema rather than by free text.

On the input side, content held as typed documents rather than rendered pages can be pinned by revision, so the example you labeled in March is the example you score in November. When the eval inputs are user utterances, screenshots, or third-party documents you do not control, the same reproducibility has to come from snapshotting instead, since there is no revision history to point at.

On the output side, a content model can express an expected label as a typed field or a reference to an existing taxonomy document rather than a free-text string. A label that must resolve to one of the taxonomy documents that already exist cannot be spelled three different ways by three different labelers, which removes one mechanical source of the label disagreement described above. This is also why the current generation of structured decision models, sometimes called System One models, is comparatively straightforward to evaluate: the model returns probabilities over a fixed answer space rather than prose, so scoring is a comparison between two members of a known set instead of a judgment about free text.

Last updated: October 7, 2026