CoursesRetrieval methods for AI agentsSorting questions by what they cost
Retrieval methods for AI agents

Sorting questions by what they cost

Run three tests over a question your agent gets asked, and decide whether your content should answer it live or whether someone should settle the answer first.
Log in to mark your progress for each Lesson and Task

How many times are you willing to pay for the same answer?

That sorts a question set faster than any comparison of features, and it runs per question rather than per project. You are deciding how your agent should answer this particular question, not which product your company adopts. A real agent's question set lands on both answers, which is the normal outcome, not a sign that something went wrong.

Three properties decide how expensive a question is:

  1. Gathering. One lookup, or a dozen?
  2. Reconciling. One source, or several that disagree across multiple systems?
  3. Judgment. A value you return, or a conclusion someone has to reach?

When none of the three are present, reach for GROQ mode and let the agent query your live content at the moment someone asks. When one or more are present on a question people ask repeatedly, settle the answer ahead of time in a knowledge base.

One more question tells you how much the cost matters: is there a single right answer, and what happens if your agent gets it wrong? A question with one correct answer and consequences attached to missing it is worth settling ahead of time even when querying it would be cheap, because settling it puts a person between your sources and your customer.

Consider a few examples:

  • Which integrations support SSO? One lookup, one source, a value that already exists. A query: *[_type == "integration" && supportsSSO == true && status == "ga"]. The catalog changes, the answer changes with it, and nothing needs rebuilding.
  • How many integrations support SCIM? Many records, but still one source and a value, computed rather than stored: count(*[_type == "integration" && supportsSCIM == true]). A query again, and here only GROQ can do the work, because a knowledge base has no way to compute over records.
  • Where is customer data stored? Four sources, actively disagreeing, with contractual exposure attached to getting it wrong. All three properties present on a question enterprise buyers ask constantly. Settle this ahead of time with a knowledge base.
  • How do I set up SAML with Okta? Setup guides, release notes, and a migration doc that describes an older flow. Gathering and reconciling both present, with version drift as the specific risk. Settle this ahead of time with a knowledge base.
  • Are we over our usage limit right now? Neither method applies. Usage belongs to your billing system, which is the only thing that knows the current number, and a later lesson covers what to do with questions like it.

The common first instinct is to ask whether GROQ can answer a question. It usually can, including the hard ones, given enough tool calls and enough time. The more useful question is what it costs you when the wrong answer reaches a customer. A question whose wrong answer means a refund you didn't owe, a contract term you can't honor, or a compliance claim you can't support is worth settling ahead of time even if a query could produce it.

Settling an answer ahead of time carries its own cost. Builds run on a refresh schedule set per knowledge base rather than per source, so a settled answer is only as current as the last build. Stable facts tolerate that well. Pricing, availability, and anything else that moves between builds does not. Settle what's stable. Query what moves.

You must be logged in to answer questions
A customer asks what your data retention policy is, and the answer turns out to be expensive to produce. Which property makes it expensive?
1
The answer requires reconciling sources that disagree
2
The query has to scan too many records
3
The answer has to reflect real-time data
4
The content model has no field for retention

Sorting works until a question lands somewhere neither method can reach.

You have 1 uncompleted question in this lesson
0 of 1