Log probabilities (logprobs) definition

Log probabilities are the model's own numbers for how likely each possible next token was, expressed as logs so that sequence probabilities can be added instead of multiplied. They are most useful when the answer space is small and fixed, and in Sanity that space usually already exists in the content model, as a string field with a predefined list of values or a reference constrained to the document types that make up a taxonomy. A score over three defined options means something; a score over free text rarely does.

What are log probabilities (logprobs) in LLMs?

Log probabilities, usually shortened to logprobs, are the logarithm of the probability a large language model assigns to a token at a given position in its output. A token is the unit a model generates, roughly a word or a word fragment. At every step, the model computes a probability for every token in its vocabulary, conditioned on everything before it, then one token is selected. The logprob is the natural log of that probability.

Because a probability sits between 0 and 1, its logarithm is always zero or negative. A logprob of 0.0 means the model gave that token a 100% probability. The closer a logprob is to zero, the more confident the model was in that token. To get back to a plain probability, exponentiate: probability = exp(logprob). A logprob of -0.0214 is about 97.9%, and -4.82 is about 0.8%.

Those two numbers come from a real worked example. In Google's guide to logprobs on Vertex AI, published July 16, 2025, a sentiment classification run with gemini-2.5-flash returns the chosen token 'Neutral' at -0.0214, with 'Positive' at -4.8219 and 'Negative' at -5.6293 as the next two candidates. Read back as probabilities, that is roughly 98% against under 1% each, which tells you the model was not torn between the labels.

Why do language models use logs instead of plain probabilities?

Language models and the tools around them work in logs for two reasons, both of which matter for logprobs specifically.

The first is that the probability of a whole sequence is the product of its per-token probabilities, and logs turn multiplication into addition. As OpenAI's cookbook on using logprobs puts it, summing the logprobs of a sequence's tokens gives you the joint log probability of that sequence, and exponentiating the sum gives the joint probability. That is how you score a complete answer rather than a single fragment of one.

The second is numerical stability. Multiplying hundreds or thousands of numbers smaller than one drives the result toward zero fast enough that standard floating-point arithmetic loses it entirely, a problem known as underflow. Adding logs keeps the value in a range a computer can represent. This is long-standing practice across statistics and machine learning rather than anything invented for language models; see log probability and the Stan user's guide on arithmetic precision.

Two derived quantities follow from the same arithmetic. Average per-token logprob lets you compare candidate generations of different lengths without penalizing the longer one. Perplexity is the exponentiated negative mean logprob, a common measure of how surprised a model is by a piece of text.

How do you get logprobs back from a model API?

Logprobs are returned as an opt-in field on a normal generation request, and the exact parameter depends on the API.

In OpenAI's Responses API, you add `message.output_text.logprobs` to the request's `include` array, described in the API reference as including logprobs with assistant messages. The companion `top_logprobs` parameter is an integer between 0 and 20 and controls how many of the most likely tokens are returned at each position, each with its log probability. OpenAI's Chat Completions API exposes a boolean `logprobs` parameter alongside `top_logprobs`.

For Gemini on Vertex AI, Google's guide sets two fields inside `GenerateContentConfig`: `response_logprobs`, a boolean that defaults to False and returns the logprob of each chosen token, and `logprobs`, an integer whose accepted value is between 1 and 20, which returns the top alternatives at each step. The result arrives on the candidate as `logprobs_result`, with `chosen_candidates` and `top_candidates`. Other platforms have added similar support; Amazon, for example, documents log probability support for Bedrock Custom Model Import.

One clarification worth making: logprobs are not logits. Logits are the raw, unbounded scores the final layer produces before the softmax function turns them into probabilities. Logprobs are what you get after the softmax and a log, so they are capped at zero. APIs return logprobs.

What do developers actually use logprobs for?

Developers use logprobs to attach a number to a model's output so that downstream code can act on it rather than treating every answer as equally trustworthy. Four uses come up repeatedly in the vendor documentation.

The first is a confidence score on a classification or a yes/no answer. OpenAI's cookbook describes logprobs as providing a probability associated with each class prediction, which lets you set classification or confidence thresholds. Google's guide calls the same pattern confidence-based thresholding.

The second is ranking. Score several candidate answers by their summed or averaged logprob and keep the highest, which is useful when you generate multiple completions and need to pick one programmatically.

The third is routing uncertain cases to a person. Google's guide names this as detecting ambiguity for human review: compare the top logprob against the second-choice logprob, and when the gap between them is smaller than a margin you choose, send the item to a reviewer instead of acting on it automatically.

The fourth is reducing confident-sounding fabrication. The cookbook's retrieval confidence pattern has the model emit a `has_sufficient_context_for_answer` boolean, then uses the logprob on that token as a score for whether the retrieved material genuinely supports the answer. The cookbook's own phrasing is careful: evaluations of this type can reduce retrieval-based hallucinations, not eliminate them.

Are logprobs the same as a confidence score?

No. A logprob tells you how likely the model considered a token at the moment it generated it. It does not tell you how often the model turns out to be correct when it produces that number, and those are different questions.

A system is calibrated when, across all the cases where it reports 90%, it is right about 90% of the time. Token probabilities are not calibrated by default, and the training process itself can move them. OpenAI's GPT-4 Technical Report reports an expected calibration error of about 0.007 for the pre-trained model on a subset of MMLU, and states that the RLHF post-training process degrades that calibration. If you want a threshold you can defend, fit it on your own labeled examples rather than assuming a raw logprob means what the number suggests.

Two further limits. A multi-token answer has one logprob per token, not one per answer, so 'San Francisco' gives you several numbers that have to be combined before you have anything describing the answer as a whole. This is why classification examples in vendor guides constrain the output to a single word.

And a model writing 'I am 90% sure' in its text is a different signal from the probability mass behind the token. The evidence here is genuinely mixed. Kadavath et al. found that models carry a usable self-evaluation signal that improves with scale, and Tian et al. found that for some RLHF-tuned models, verbalized confidence elicited by prompting can be better calibrated than the model's own conditional probabilities. Which signal is stronger depends on the model and on how you ask.

How do logprobs relate to decision models?

Decision models shortcut most of the work logprobs are used for, because they return a choice from a fixed, developer-supplied answer space rather than free text. The probability mass is distributed over the answers themselves, so there is no multi-token answer to stitch together and no prompt engineering required to keep the output to one word.

OpenAI announced its Decisions API at DevDay on September 29, 2026, describing it in the DevDay 2026 recap as enabling real-time decision-making by focusing Luna's intelligence on a specific set of user-defined questions with finite pre-defined answers, with developers supplying context as text or images and getting back answers they can use to classify content, route requests, or choose an agent's next action. It is powered by GPT-6 Luna, released September 22, 2026. TypeSafe AI's Jev, released September 15, 2026, was the first model sold as a System One model in the same broad category.

What does not change is the input side. A score over a defined answer space is only as meaningful as the answer space, and that is a content modeling problem. Sanity is the AI Content Operating System, and the structure it holds is what gives a model something bounded to decide over: typed fields, a string field whose `options.list` holds a static array of predefined values such as ['sci-fi', 'western'] or {title, value} objects, and reference fields whose `to` array names exactly which document types are valid targets, so a taxonomy stays a closed set of real documents rather than a free-text label.

Last updated: October 5, 2026