Contrastive Language Model (CLM) definition
A contrastive language model (CLM) is a model that scores how well a candidate action fits a given state by comparing embeddings from two separately trained encoders, a state encoder and an action encoder, instead of generating text. The encoders are trained with a contrastive objective so each state sits close to the action actually taken and far from the alternatives.
A contrastive language model judges rather than writes: it embeds a state with one encoder, embeds each candidate action with another, and ranks the candidates by how closely their embeddings align. Because the two sides are embedded separately, a fixed candidate set can be embedded once and reused while the state keeps changing. That fixed set is a content-modeling property first, and in Sanity taxonomy terms, routing destinations, and approved components are documents you can enumerate with a single GROQ query.
A contrastive language model scores candidate actions against a state by comparing embeddings, while a causal language model predicts the next token in a sequence. Both are abbreviated CLM, and they are close to opposites in what they do. A causal language model is the standard GPT-style architecture that writes prose, code, or answers one token at a time. A contrastive language model writes nothing at all. You hand it a description of the current situation (the state) and a list of things that could be done next (the candidate actions), and it returns those candidates ranked by fit.
The mechanism is a dual encoder. One encoder turns the state into a vector, a second encoder turns each candidate action into a vector, and the score is the alignment between them. The training objective is contrastive: for every training example, the model pulls the state's embedding toward the ground-truth action and pushes it away from every other action in the batch. The specific loss named for the released model is InfoNCE, a bidirectional version of it.
If that recipe sounds familiar, it should. It is the same dual-encoder contrastive setup behind image-text models like CLIP and behind dense retrieval, where a query encoder and a document encoder are trained so that matching pairs land near each other. A contrastive language model applies it to states and actions rather than to image-caption or query-document pairs. Dual encoders trained this way are a well-studied family; see, for example, Dual-Encoders for Extreme Multi-label Classification (ICLR 2024).
See Jev turn text into structured content
Vellum is a Sanity experiment that uses Jev to turn loose text into structured content. Paste any text, pick a Sanity, Zod or JSON Schema, and Jev decides which part of the source fills each field. Anything below your confidence threshold stays empty for review.
CLM-8B is the first released contrastive language model, announced on 23 September 2026 by Jacky Kwok as joint work with Hazy Research. The launch post describes it as "an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions." It belongs to the System One model class, models that judge rather than write, alongside TypeSafe's Jev.
Architecturally, CLM-8B is lighter than the name suggests. Per the Hugging Face model card, the 8B refers to a frozen Qwen3-8B encoder; what is actually trained is two small projection heads, one for states and one for actions, sitting on top of it. The heads are roughly 20M trainable parameters each, and the reference head checkpoint published in the GitHub repository is about 75 MB. Because only the heads train, the card notes that fine-tuning is cheap. The weights are Apache 2.0, as is the Qwen3-8B base, so the model is self-hostable.
The training recipe reported on both the README and the model card runs in three stages: pre-training on 60M Nemotron question-and-answer pairs, mid-training on 30M synthetic hard negatives, and post-training on 1M agentic trajectories. The authors say a multimodal CLM-35B is in training, with release announced for early October 2026.
A contrastive language model is fast because the state and the actions are embedded independently, so their embeddings can be cached and reused separately. The authors call this disaggregating states and actions. A generative judge has to read the state and the candidates together on every call. A contrastive language model does not: if the action set is fixed and only the state changes, you embed each action once, keep those vectors, and pay for a single fresh embedding of the new state plus one cheap similarity comparison per cached candidate.
That is why the speedup grows with the number of candidates. The Hugging Face model card reports that with around 1,000 candidates, CLM is 13x faster than Jev. The authors present this as the structural advantage over Jev, which they describe as supporting state caching only.
The condition matters more than the number. The caching win requires a bounded, stable, enumerable set of candidate actions: a fixed menu of tools, a known taxonomy, a finite list of routing destinations. If the candidate set is regenerated from scratch on every call, there is nothing to cache and the advantage disappears.
CLM and Jev are both System One models that score candidates rather than generate text, but they differ in architecture, licensing, and what can be cached. Jev, per TypeSafe's published description, is a parallel sampler that computes all outputs in a single pass, evaluating each question independently against one shared read of the state, trained with Reinforcement Learning for Calibrated Decisions, with closed weights and hosted-only access. CLM is a dual-encoder contrastive model with open Apache 2.0 weights that you run yourself, and its two sides cache independently.
Every performance comparison below is self-reported by the CLM authors and has not been independently reproduced. The Jev figures in their tables are a competitor's measurements of Jev; Jev's own published numbers are TypeSafe's. The authors report zero-shot performance on par with Jev across computer-use, gaming, and tool-calling tasks with up to 9x lower latency. The 9x figure is the best row, not the average: per the task table reproduced by explainx, T-Rex runs 16.5 ms versus 149.8 ms (about 9.1x), Super Mario about 4.0x, WikiRacing about 2.8x, and tool calling on BFCL V4 about 1.6x. On two of those tasks Jev is slightly more accurate: 99.2% versus 95.2% on tool calling, and 30/30 versus 26/30 on WikiRacing. Several of these runs involve only 5 or 30 trials, so treat the success columns as directional rather than settled.
After lightweight fine-tuning of the heads, the README reports state-of-the-art verifier results: 81.6% on DeepSWE (best of 4 candidates, 38 held-out tasks, against a 73.7% pass@1 baseline) and 87.6% on Terminal-Bench 2.1 (best of 5, 30 held-out tasks, against an 84.0% baseline), with verifier latency of 79 ms and 32 ms respectively on an H100. In the same tests the authors report Jev scoring below the random-selection baseline, 71.1% and 83.1%, and state that Jev fails to serve as a verifier for these long-horizon tasks. The authors also report that test contrastive loss follows a power law in training compute, dataset size, projection-head size, and encoder size, which must be scaled jointly.
A contrastive language model cannot generate. The model card states plainly that CLM only scores the candidates you give it, so it produces no prose, no rationale, no draft, and no code. If you need the reasoning rather than the verdict, it is the wrong tool, and if you cannot enumerate the answer space in advance, it has nothing to rank.
Its scores are also not calibrated probabilities. The softmax it returns is normalized across the candidate set you supplied, and the card says the probabilities are relative to that set. Change the candidates and the numbers change. Nothing in the announcement claims those scores match observed frequencies, which is what calibration actually means, so do not read a 0.9 as a nine-in-ten chance of being right.
Two further limits come from the authors themselves. The model is encoder-locked: the trained heads require Qwen3-8B last-token-pooled embeddings, so you cannot swap the backbone. And the headline agentic coding results come from fine-tuned heads, not from the released checkpoint used zero-shot, so the base model and the benchmark-topping configuration are not the same thing.
Structured content is what makes a candidate set enumerable, which is the precondition a contrastive language model needs before any of its caching advantage applies. A rendered web page is one blob of markup: you cannot point at the field a question concerns, and you cannot list the legal next actions without guessing. Typed content solves both halves. Content Lake stores content as discrete typed fields and blocks rather than as rendered output, so the state you score against can be the exact block or field in question. A GROQ projection returns just those fields, which keeps the state clean and small.
The candidate side works the same way. Reference fields point at a finite set of documents, and that set is itself a query: a taxonomy of category documents, a list of routing destinations, or a roster of approved components can be pulled with a single GROQ query, embedded once, and reused across thousands of scoring calls while the state keeps changing. Because schemas are defined in code, the set of document types and allowed values is a known, versioned list rather than something discovered at runtime. That is the regime where disaggregated caching pays off.
Sanity is the AI Content Operating System, the intelligent back end for companies building AI content operations at scale, and the relationship to contrastive scoring is architectural. What structured content provides is the bounded, well-defined action space that this class of model requires in order to be useful at all.
Discover More with Sanity
Now that you've learned about Contrastive Language Model, why not start exploring what Sanity has to offer? Dive into our platform and see how it can support your content needs.
Last updated: