Sorting questions by what they cost
How many times are you willing to pay for the same answer?
That sorts a question set faster than any comparison of features, and it runs per question rather than per project. You are deciding how your agent should answer this particular question, not which product your company adopts. A real agent's question set lands on both answers, which is the normal outcome, not a sign that something went wrong.
Three properties decide how expensive a question is:
- Gathering. One lookup, or a dozen?
- Reconciling. One source, or several that disagree across multiple systems?
- Judgment. A value you return, or a conclusion someone has to reach?
When none of the three are present, reach for GROQ mode and let the agent query your live content at the moment someone asks. When one or more are present on a question people ask repeatedly, settle the answer ahead of time in a knowledge base.
One more question tells you how much the cost matters: is there a single right answer, and what happens if your agent gets it wrong? A question with one correct answer and consequences attached to missing it is worth settling ahead of time even when querying it would be cheap, because settling it puts a person between your sources and your customer.
Consider a few examples:
- Which integrations support SSO? One lookup, one source, a value that already exists. A query:
*[_type == "integration" && supportsSSO == true && status == "ga"]. The catalog changes, the answer changes with it, and nothing needs rebuilding. - How many integrations support SCIM? Many records, but still one source and a value, computed rather than stored:
count(*[_type == "integration" && supportsSCIM == true]). A query again, and here only GROQ can do the work, because a knowledge base has no way to compute over records. - Where is customer data stored? Four sources, actively disagreeing, with contractual exposure attached to getting it wrong. All three properties present on a question enterprise buyers ask constantly. Settle this ahead of time with a knowledge base.
- How do I set up SAML with Okta? Setup guides, release notes, and a migration doc that describes an older flow. Gathering and reconciling both present, with version drift as the specific risk. Settle this ahead of time with a knowledge base.
- Are we over our usage limit right now? Neither method applies. Usage belongs to your billing system, which is the only thing that knows the current number, and a later lesson covers what to do with questions like it.
The common first instinct is to ask whether GROQ can answer a question. It usually can, including the hard ones, given enough tool calls and enough time. The more useful question is what it costs you when the wrong answer reaches a customer. A question whose wrong answer means a refund you didn't owe, a contract term you can't honor, or a compliance claim you can't support is worth settling ahead of time even if a query could produce it.
Settling an answer ahead of time carries its own cost. Builds run on a refresh schedule set per knowledge base rather than per source, so a settled answer is only as current as the last build. Stable facts tolerate that well. Pricing, availability, and anything else that moves between builds does not. Settle what's stable. Query what moves.
Sorting works until a question lands somewhere neither method can reach.