AI guardrails definition
AI guardrails are the controls that keep an AI system operating inside defined boundaries: the policies, technical checks, and monitoring that constrain what goes into a model, what the model is allowed to do, and what is allowed back out to a user. They sit outside the model rather than in its weights, so they can be改
AI guardrails are the runtime controls that keep an AI system inside defined boundaries, covering what reaches the model, what it may do, and what reaches the user. Half of that is classifiers; the other half is policy someone has to own. In Sanity, that policy lives as content: an agent's never-say list, escalation rules, and voice are fields on a document owned by Compliance, Support, and Brand, versioned and rolled back like any other content.
AI guardrails are usually sorted one of two ways: by where they sit in the stack, or by when they run during a request.
IBM's September 2025 taxonomy splits them by layer. Data guardrails govern training and reference inputs, including cleansed datasets and privacy rules. Model guardrails cover fine-tuning, validation, and monitoring against metrics like toxicity and accuracy. Application guardrails are the policies an API enforces on what a chatbot may say or do. Infrastructure guardrails are access controls, encryption, and logging. IBM calls AI governance the "zeroth" guardrail, the layer that keeps the other four consistent across a business.
NVIDIA's NeMo Guardrails documentation splits them by stage of the request instead, which is the more concrete view. Input rails run before the model and handle content safety, jailbreak detection, topic control, and masking of personal data. Retrieval rails filter the documents and chunks that are allowed to become context. Dialog rails constrain how a multi-turn conversation can flow. Execution rails validate tool and function calls, their arguments, and their results. Output rails filter, edit, or block the response before a user sees it. NVIDIA notes that input and output rails are the most common in practice.
Both taxonomies describe the same system from different angles, and a production deployment typically uses several rails at once rather than picking one.
AI guardrails are external and run at request time, which separates them from the three concepts they are often confused with.
Alignment (reinforcement learning from human feedback, safety fine-tuning) changes the model's behavior by changing its weights. AI guardrails sit outside the model, which is why you can change them without retraining, and why they still apply when you swap one model for another.
Evals measure behavior before you ship. AI guardrails act on a live request after you have shipped. They complement each other: the eval suite tells you whether the guardrail works, and the guardrail is what blocks the bad turn.
AI governance is the policy and accountability layer, deciding who sets the rules for what the system may do, and how those decisions are evidenced. AI guardrails are the enforcement of those decisions in the running system. NIST's AI Risk Management Framework describes this policy layer; IBM's "zeroth guardrail" framing says the same thing more bluntly.
One more distinction is worth stating plainly. A "never say X" line in a system prompt is a soft instruction, not an enforced check. Under adversarial pressure, the model may ignore it. A guardrail is an enforced check outside the model. Plenty of teams believe they have guardrails when what they have is a paragraph of English.
A content filter is one kind of AI guardrail, usually an output classifier scanning for toxicity or personal data, but it does nothing about what the model is allowed to reach or do.
The broader category includes topic control, schema validation of structured output, tool-use validation, grounding checks against retrieved sources, and identity boundaries. OWASP's Top 10 for LLM Applications recommends guardrails external to the model (validation, filtering, and policy enforcement) as a defense against prompt injection and data leakage, which no output filter catches on its own.
The strongest evidence for looking past filters is about access, not language. IBM's Cost of a Data Breach Report 2025, cited on its AI guardrails page, found that 97% of AI-related breaches occurred in an environment without access controls, against an average US breach cost of $10.22 million in 2025. That points to the identity guardrail, not the classifier.
Agent architectures raise the same issue from another direction. Tool descriptions are trusted text: if you install a Model Context Protocol (MCP) server, its tool descriptions land in the prompt every turn, and a sloppy or malicious one can prompt-inject the agent before the user has typed anything.
AI guardrail policy goes stale when it is a string in a repository that only engineers can edit. The fix is to give each part of the policy an owner who can change it without filing a pull request.
In Sanity, the agent's system prompt is modeled as a document type with the policy split into fields by owner: voice is owned by Brand, the way the agent uses user context is owned by Product, escalation rules are owned by Support, and the forbidden topics list is owned by Compliance. Splitting the prompt into fields is access control, not cosmetics. Because it is content in the Studio, the policy gets real-time collaboration, version history, scheduled publishing, and rollback for free. Nearform put it this way: storing the system prompt in a Sanity document is genuinely useful, and editors tuned the agent's voice without any code changes.
The obvious objection is what stops a non-engineer from breaking the agent. The answer is an eval bench: a frozen set of representative conversations, twenty to start, each scored against a rubric you wrote, run on every model change, prompt change, and tool change. The bar to ship anything is the bench staying green. Author it like content, and gate it like code.
This is one expression of Sanity as the Content Operating System for the AI era: the governance you already use for a website, meaning drafts, scheduling, history, permission gating, and audit trails, applied to what an agent is allowed to say.
A production AI guardrail setup combines runtime checks, an identity boundary, a loop limit, and a failure taxonomy that tells you which layer to fix.
The identity boundary is the one most often missed. The agent's reach should be the user's reach, which means forwarding the user's token into every tool call so the agent inherits your existing security model: the same row-level permissions, the same rate limits, and the same regulatory boundaries. You get personalized retrieval (the agent sees what the user can see), personalized action (it does what the user could do), and traceable audit (the action is logged against the user, not the model). You do not build AI security as a separate discipline. You make sure the token flows.
The loop limit is cheap and concrete. Agent loops carry a maximum iteration count, usually somewhere between 20 and 50, so a stuck agent cannot run forever burning tokens.
Finally, tag failures by layer, not by severity. Scope-violation means the never-say list should have caught it, so the fix is in the prompt. Auth-confusion means the agent acted under the wrong identity, so the fix is in tools and auth. Prompt-drift means a change shipped without the eval bench catching the regression. Hallucination usually means retrieval returned nothing useful and the model filled the gap, which is a retrieval problem, not a model problem. Each tag points to a layer, and the fix belongs at that layer.
Discover More with Sanity
Now that you've learned about AI guardrails, why not start exploring what Sanity has to offer? Dive into our platform and see how it can support your content needs.
Last updated: