Hybrid search definition

Hybrid search combines keyword matching with vector similarity and fuses the two rankings into one result list, which catches both the query that names an exact product code and the query that describes an idea in words the document never uses. In Sanity, the same structured document can carry an embedding for the semantic half and typed fields such as locale, status, and category for the lexical half and for filtering, so both retrieval paths read from one source rather than two drifting copies.

Why is hybrid search better than semantic search alone?

Hybrid search is better than semantic search alone because vector similarity is weak on queries that need to be literal. A vector search turns the query and the documents into numeric representations called embeddings and returns the documents whose vectors sit closest to the query's. That works well for "how do I cancel my plan" matching a page titled "Ending your subscription". It works poorly for "error TS2345", a part number, a customer name, or a version string, because those tokens carry little semantic meaning and an embedding model can return something that is only in the same subject area.

Keyword search has the mirror-image problem. Classic lexical search, usually scored with an algorithm called BM25, matches the terms a person actually typed. If the document says "invoice" and the searcher typed "bill", a pure keyword index returns nothing useful no matter how good the document is.

Hybrid search runs both and merges them, so a rare exact token is found by the keyword side and a paraphrase is found by the vector side. The pattern we see in production retrieval is that teams reach for hybrid not because semantic search failed generally, but because it failed on a specific, visible, embarrassing class of query, identifiers, names, and codes.

How does hybrid search combine keyword and vector results?

Hybrid search combines the two result sets by score fusion, and the most common method is Reciprocal Rank Fusion (RRF), which ignores the raw scores entirely and uses only each document's position in each list. The reason it ignores raw scores is practical: a BM25 score and a cosine similarity are on completely different scales, and normalizing them against each other is fragile.

RRF works in four steps. First, run the query through the keyword index and get a ranked list. Second, run the same query through the vector index and get a second ranked list. Third, give every document a score of 1 divided by (k plus its rank) in each list, where k is a small constant, commonly 60 in the original RRF paper by Cormack, Clarke, and Buettcher. Fourth, add each document's contributions together and sort by the total. A document ranked third in both lists beats a document ranked first in one and absent from the other, which is the behavior you want when one retriever is confidently wrong.

The alternative is a weighted sum of normalized scores, where you pick a weight, say 0.7 semantic and 0.3 lexical. It gives you a tuning dial, which helps when you know your query mix, and it creates a tuning burden, which is a real cost when you do not.

What is reranking, and is it part of hybrid search?

Reranking is a second scoring pass applied after hybrid search retrieval, and it is a separate stage rather than part of the fusion itself. Hybrid search's job is recall, pull perhaps the top 50 candidates from the keyword and vector indexes so that the right document is somewhere in the pile. A reranker, usually a cross-encoder model that reads the query and each candidate document together rather than comparing precomputed vectors, then reorders those 50 and hands the top 5 to whatever consumes them.

The division of labor matters because cross-encoders are accurate and slow. Scoring every document in a collection with one is not viable, but scoring 50 is. In the retrieval systems we have worked on, the useful mental model is that hybrid search decides what gets considered, and the reranker decides what gets used.

You can ship hybrid search without a reranker and get most of the benefit on a small corpus. As the corpus grows and the top 50 fills up with near-duplicates, the reranking stage is usually what recovers precision.

Why does hybrid search need metadata filters as well?

Hybrid search needs metadata filters because neither the keyword score nor the vector score understands correctness constraints like language, permission, or publication state. A filter is a hard pre-condition applied to the candidate set: only documents in French, only documents the requesting user is allowed to see, only documents currently published. Relevance ranking cannot express that, because a highly relevant document in the wrong locale still scores high and still ships the wrong answer.

This is where the shape of the underlying content decides how good the search can get. If locale, status, product line, and effective date live as typed fields on the document, they are available as filters and as facets. If they live only inside the prose, or only in a filename convention, you cannot filter on them reliably and each constraint has to be re-derived at query time.

Structured content is what makes that half of hybrid search tractable, and it is the pillar Sanity is built around: the Content Operating System for the AI era, where the fields that govern an editorial workflow are the same fields a retrieval layer filters on. Freshness belongs in the same category of constraint. An index built from a nightly export can serve a confidently stale answer, which is why teams running retrieval against live content tend to re-index on the publish event rather than on a schedule.

When is hybrid search worth the extra complexity?

Hybrid search is worth the extra complexity when your query log contains both natural-language questions and literal identifiers, and it is usually not worth it when it contains only one of those. Running hybrid search means maintaining two indexes, keeping them in sync with the same source content, and owning a fusion step that has to be evaluated rather than assumed.

Three situations where it earns that cost, from the retrieval work we have seen: documentation and support search, where users type both "why is my build failing" and a literal error code; ecommerce, where SKUs, model numbers, and brand names sit alongside descriptive queries like "warm jacket for hiking"; and retrieval-augmented generation (RAG), where the passages you feed a language model determine the answer it produces, so a miss on recall becomes a wrong answer rather than an empty results page.

If your content is short, uniform, and always queried in natural language, semantic search alone may be enough. The honest test is to assemble 50 real queries from your logs, including the ugly ones, and check whether the correct document appears in the top 5 for each retrieval method. Hybrid search is worth adopting when it wins on that set, not because it is the more sophisticated architecture.

Last updated: September 23, 2026