Guardrails AI, or checking what the model answered

Guardrails AI, or checking what the model answered

A language model answers with text, and your application needs something specific: the right structure, no personal data, content staying within accepted bounds. The gap between those two things is where problems arise.

Guardrails AI fills that gap. It is a library wrapping a model call in a layer that checks the result, with the option of correcting it or asking the model for another attempt.

Its construction differs from tools steering conversation flow, covered in the piece on NeMo Guardrails. There you had an intermediate layer conducting a dialogue; here you have a library wired into a specific call and focused on its result.

Guards and validators

The basic concept is a guard, meaning a set of checks placed over a model call. You assemble checks from ready pieces or write your own.

from guardrails import Guard
from guardrails.hub import DetectPII, ToxicLanguage

guard = Guard().use(
    DetectPII(pii_entities=["EMAIL_ADDRESS", "PHONE_NUMBER"], on_fail="fix"),
    ToxicLanguage(threshold=0.8, on_fail="exception"),
)

result = guard(
    model.chat.completions.create,
    model="gpt-4o",
    messages=[{"role": "user", "content": question}],
)

The catalogue of ready validators covers things that take weeks to implement yourself: personal data detection, toxicity scoring, pattern matching, detecting competitor mentions, restricting topic scope, and detecting attempts to circumvent instructions.

That catalogue's value lies in each validator being a separate package installed on demand. You do not pull toxicity detection models into a project that only checks response structure.

The way you install them changed during 2026, which is worth knowing before you copy an older example. Validators are now ordinary packages on the public PyPI index, named with a guardrails-ai- prefix, while the former private registry driven by its own command, along with the servers on which some validators ran their models remotely, were shut down on the sixth of August 2026. Models therefore run locally or on an endpoint you host yourself, and the previous import path stayed for a while as a compatibility shim.

Note, though, that validators differ in cost by orders of magnitude. A pattern check is free, while personal data detection or toxicity scoring runs its own model, adding time and resources.

Failure actions

This is the most interesting part of the construction and where deliberate decisions pay off, since cost follows from them.

The first option is ignoring and recording the fact in a log. Useful during rollout, when you want to see how often a check would have fired before it starts blocking anything.

The second is correcting deterministically. A detected email address gets masked, overly long text trimmed. Cheap and predictable, provided the correction can be made mechanically.

The third is filtering out the offending fragment while keeping the rest.

The fourth is refusing with a prepared response instead of what the model said.

The fifth, the most interesting and most expensive, is asking the model for another attempt with an explanation of what was wrong. The model receives its own answer along with the validator's message and generates a new one.

That last one deserves separate treatment, since its cost gets underestimated.

The cost of re asking

The retry mechanism looks elegant and carries a price to work out before enabling it broadly.

On failure you pay for a full generation, and then for another. If the second attempt also fails, you pay a third time. At ten percent of answers needing correction the bill rises by a dozen or so percent; at fifty percent it almost doubles.

Time joins that. A retry means the full latency of another call, so an answer that took two seconds takes four with one correction.

The practical conclusion has two parts. First, set an attempt limit and decide what happens once it is exhausted, since the default is rarely what you want in production. Second, treat a high retry count as a signal rather than a normal state.

If the model regularly misses the required response shape, the cheaper answer is fixing the instructions or using structure enforcement on the API side, rather than generating an answer twice.

That last point deserves emphasis, since it changed in recent years. Models now accept a response schema and enforce it during generation, so validating structure after the fact became largely redundant. This library's value shifted towards checks a model will not perform itself: personal data, topic scope, and business rules.

Response structure and schemas

Historically the library relied on its own structure description format written as markup. That still works and turns up in older material.

The more convenient route now is describing the expected structure with an ordinary data model you already have in the project.

from pydantic import BaseModel, Field

class Ticket(BaseModel):
    category: str = Field(description="one of: complaint, question, outage")
    urgency: int = Field(ge=1, le=5)
    summary: str

guard = Guard.for_pydantic(Ticket)

The advantage is obvious: the same description serves validation, editor completion, and type checking across the rest of the application, so there are not two places that must stay in agreement.

The caveat from the previous section applies here too. If your model vendor supports structure enforcement, use it and leave the library the checks a model will not perform: value ranges, business rules, and dependencies between fields.

Checking input, not only output

The name and most material direct attention to the model's answer, while a guard can also cover what reaches it. That is often more important and is cheaper.

Checking input catches three things. The first is data that should not leave your infrastructure: a user pastes a contract excerpt with names in it, or an access key, to ask what is wrong. Catching that before sending is the only moment when reacting is possible.

The second is attempts to circumvent instructions. Catching them on input costs a fraction of generating an answer and checking it afterwards, since the model is never called.

The third is out of scope questions. A refusal issued before the model call is free, while a refusal after generating an answer costs a full call and as much time as an answer given.

From that follows a rule worth adopting: move checks as early as possible. Every one that can run on input saves a model call rather than merely adding safety.

The exceptions are checks that by nature concern the result: response structure, personal data the model generated, agreement with facts from documentation. Those cannot be moved earlier and there the cost is unavoidable.

Custom validators

The catalogue covers general cases, and business rules are by definition specific, so sooner or later you will write your own validator.

from guardrails.validators import Validator, register_validator, PassResult, FailResult

@register_validator(name="in-price-list", data_type="string")
class InPriceList(Validator):
    def validate(self, value, metadata):
        if value not in metadata["available_plans"]:
            return FailResult(
                error_message=f"Plan {value} does not exist in the price list.",
                fix_value=metadata["available_plans"][0],
            )
        return PassResult()

Three things deserve getting right when writing your own check.

The first is the error message. Under the re asking action that message reaches the model as guidance, so a sentence stating plainly what is wrong and what you expect raises the chance of a correct second attempt more than any change to the system instruction.

The second is the fallback value. If a correct value can be supplied mechanically, supply it, since a deterministic correction is free while re asking costs a full call.

The third is speed. A validator running on every answer should complete in milliseconds. A check querying a database or an external service adds latency to every answer, so consider caching the data it relies on.

Guardrails AI against the alternatives

Consider the third row first, since for response structure alone it solves the problem without adding anything. The library starts paying off where checks reach beyond shape: personal data, conversation topic, and compliance with organisational rules.

Choosing between the first and second rows comes down to what you control. A single call's result is the first row. A whole conversation's flow is the second.

Remember too that libraries oriented towards typed results, such as the one covered in the piece on Pydantic AI, solve part of this problem incidentally, without a separate validation layer.

Deployment practice

A few things that when running this in production separate a working solution from one generating costs and noise.

Start in observation mode. Enable validators with their action limited to logging and spend two weeks gathering data on how often each would have fired. That number usually surprises in both directions: some checks never fire, others would block every tenth answer.

Then enable them one at a time, starting with the cheapest. Pattern based checks cost little, so they can run on every call. Checks running their own model deserve limiting to cases where they are genuinely needed.

Record the cases where validation fired. That is the only way to distinguish a correct block from a false alarm, and false alarms with this class of tool damage a product more than letting content through.

Plan behaviour when the validation layer itself fails too. A validator using a model can stop responding, and two behaviours make sense then: passing traffic through unchecked or refusing service. The choice depends on which is worse in your case, and the default deserves checking rather than assuming.

One last matter concerns streaming responses. Validation by nature needs the whole result, so a response checked after generation cannot appear on screen gradually. That is a real trade off: either the user sees text from the first word and gets it unchecked, or waits for the whole thing and gets a verified result. On short answers the difference goes unnoticed; on long ones it decides how the product feels, and it deserves settling deliberately rather than discovering after launch.

Common mistakes

The first is validating structure after the fact when the model can enforce it during generation. That is an extra layer solving a problem that no longer exists.

The second is re asking without an attempt limit. Every attempt is a full model call, so a loop on a hard case can cost a multiple of an ordinary answer.

The third is enabling every validator at once in production. Without an observation period you do not know which will fire and which will merely add latency.

The fourth is treating a high correction count as normal. It signals that the instructions or the schema need fixing rather than that the validation layer works well.

The fifth is model based checks on every call without need. They cost many times more than pattern checks and deserve running selectively.

The sixth is not recording the cases where checks fired. Without that you cannot distinguish a correct block from a false alarm, and that is the only measure of effectiveness.

The seventh is checking responses only, when part of the rules can apply to the input. A refusal issued before the model call is free, while one issued after generation costs a full call.

FAQ

How does it differ from NeMo Guardrails?

In shape. This library wires into a specific model call and checks its result. NeMo Guardrails is an intermediate layer steering a whole conversation. The first is lighter to deploy, the second gives control over the dialogue.

What does re asking the model cost?

A full call every time. At ten percent of answers needing correction the bill rises by a dozen or so percent, and response time in those cases doubles. Set an attempt limit and treat a high correction count as a signal to change the instructions.

Do I need this when the model enforces structure?

For structure alone usually not. This library's value lies in checks a model will not perform: detecting personal data, restricting topic scope, and business rules specific to your context.

Which validators are available?

The catalogue covers personal data detection, toxicity scoring, pattern matching, detecting competitor mentions, restricting topic scope, and detecting attempts to circumvent instructions. Each is a separate package installed on demand.

How do I start safely?

In observation mode: enable validators with their action limited to logging and spend two weeks gathering data on how often each would have fired. Only then enable blocking, starting with the cheapest and most accurate checks.

Documentation sits on the project site, and the validator catalogue in a separate section.