Outlines, enforced structure for model output

Outlines, enforced structure for model output

Outlines does not ask a model for valid JSON, it removes the model's ability to produce anything else. At every decoding step it masks the tokens that would break the schema, so the output is correct by construction rather than through successful persuasion. Version 1.3.3 was released on 6 August 2026 under Apache 2.0.

How enforcing differs from asking

The usual approach to structured responses looks like this: you send a prompt describing the schema, you get text back, you try to parse it, and if that fails you ask again. That is the ask-and-check loop. At any moment the model can emit an introductory sentence, a triple backtick with a language tag, or a trailing comma in an array, and you only find out afterwards.

Outlines goes one level down. It turns the schema into a regular expression, the regular expression into a finite automaton, and the automaton into an object called a logits processor. That object receives the probability vector over the whole vocabulary before every decoding step and zeroes out the positions that the current automaton state disallows. If the automaton sits right after the "total" key and its colon, then every token other than a digit, a minus sign or a space has probability zero. The model physically has nothing else to pick.

There are two consequences and both need to be understood before you commit. The first is good: no path exists where the output fails to parse against the schema, so a whole class of errors and all the retry handling code disappear. The second is less pleasant: the guarantee is purely syntactic. A model forced to fill the currency field will put three letters there, but nobody promises they are the letters of the right currency. Enforced structure does not turn a hallucination into truth, it only gives it a regular shape.

Building the automaton is the job of one of three backends. The default for JSON schemas and for regular expressions is outlines_core, the companion Rust package, pinned exactly to version 0.2.14. For context-free grammars the default is llguidance. The third option is xgrammar. You pick a backend with the backend argument of the Generator constructor, and the constants JSON_SCHEMA_DEFAULT_BACKEND, REGEX_DEFAULT_BACKEND and CFG_DEFAULT_BACKEND in the outlines.backends module state plainly what gets chosen when you pass nothing.

Compiling the automaton costs time. The docstring of the SteerableGenerator class says so directly: the processor can be quite expensive to build, which is why the generator stores it. The practical conclusion is that you create a Generator once per schema and keep it, rather than building one inside a request loop. There is also an on-disk cache, by default in HOMEDIR/.cache/outlines, controlled by the OUTLINES_CACHE_DIR variable and secondarily by XDG_CACHE_HOME. The functions outlines.clear_cache, outlines.disable_cache and outlines.get_cache are exposed at the top level of the package.

Steerable models and black box models

This is the heart of the library and the most common misunderstanding. Token masking requires access to the probability distribution, and therefore to the model weights or to an engine that exposes such access. So Outlines splits its integrations into two groups, and it does so in code, not in the documentation.

In outlines/models/__init__.py the SteerableModel type is the union of exactly three classes: LlamaCpp, MLXLM and Transformers. Only for those does Outlines build the logits processor itself and enforce the schema itself. The BlackBoxModel type covers Anthropic, Dottxt, Gemini, LMStudio, Ollama, OpenAI, Mistral, SGLang, TGI, VLLM and VLLMOffline, and AsyncBlackBoxModel adds asynchronous variants of eight of them. For that second group the output type is not compiled locally, it is forwarded to the provider in whatever form the provider understands.

The Generator factory checks this with isinstance and returns a different class: SteerableGenerator or BlackBoxGenerator. If you try to pass your own processor through the processor argument to a model from the second group, you get a NotImplementedError saying that this model does not support logits processors. That is an honest way to state the situation, but you have to notice it before you deploy.

What happens with closed providers is clearest in the format_output_type method of the OpenAI adapter. A Pydantic model becomes a response_format of type json_schema with strict: true and a forced additionalProperties: false. A plain dict gives json_object mode. A Regex type, however, ends in a TypeError explaining that neither regex-based outputs nor the pattern keyword in a JSON Schema are available with OpenAI, and a CFG type ends in an exception about missing grammar support. Ollama behaves the same way and also rejects Regex and CFG. The Dottxt model, the commercial API of the company behind the library, rejects both with a message saying they will be available soon.

The conclusion is simple. If your provider is OpenAI or Anthropic, Outlines gives you no guarantee beyond the one the provider already gives. It is then a layer that unifies the interface, not a mechanism that enforces anything. The guarantee that comes from token masking appears only with from_transformers, from_llamacpp and from_mlxlm, that is, when the weights sit inside your process.

A separate trap concerns vLLM. The server adapter sends constraints in the structured_outputs field inside extra_body. A comment in the code says this field requires a vLLM server of at least version 0.12, which replaced the older guided_json, guided_regex and guided_grammar keys, and that older servers simply ignore the new field and return unconstrained output. Silent degradation into no guarantee is worse than an error, because nothing shows up in the logs. The offline variant VLLMOffline builds StructuredOutputsParams and puts them into SamplingParams, yet still counts as a black box, because the enforcement is done by the vLLM engine, not by Outlines.

Version 1.0 and the break with the v0 interface

Version 1.0.0 reached PyPI on 18 June 2025, and the last release from the zero branch, 0.2.3, dates from 3 April 2025. The release_note.md file shipped inside the published package explains the intent: the new version narrows Outlines down to constrained generation itself and hands the rest of the work to inference libraries and to the user.

Four changes break everything written before. First, the loader functions from the models module, such as models.transformers or models.openai, were replaced by counterparts prefixed with from_, which take an already built engine or client instance instead of a model name and a parameter dictionary. Second, the generate module together with generate.json and generate.choice disappeared in favour of a single Generator constructor that accepts any supported output type. Third, TransformersVision gave way to TransformersMultiModal, loaded through from_transformers with a processor instead of a tokenizer. Fourth, the Function class was replaced by Application, which takes no model at construction time, only at call time, and reads template variables from a dictionary rather than from keyword arguments. The ExLlamaV2 integration was removed with no replacement.

The deprecated pieces kept working with a warning only until version 1.1.0, released on 10 July 2025, less than four weeks after 1.0.0. Every tutorial written before mid-2025 shows code that will not run at all today. When you look for examples online this is a real cost, because there are still more old posts than new ones.

The release pace is uneven. After 1.2.0 on 31 July 2025 came a long series of patch releases, and 1.3.0 only appeared on 13 May 2026. From there it tightened up: 1.3.1 on 30 June, 1.3.2 on 20 July and 1.3.3 on 6 August 2026. One small inconsistency in the release feed: an entry titled Outlines v1.2.12 appears in it twice, with timestamps from 3 March and 4 May 2026. The PyPI date for 1.3.0 matches the release feed date to the day, so the numbers themselves are trustworthy.

Installation, dependencies and licence

The core of the library pulls in no engine at all. The base dependencies are jinja2, cloudpickle, diskcache, pydantic from 2.0, jsonschema, pillow, typing_extensions, genson and outlines_core pinned exactly to 0.2.14, released on 9 January 2026, also under Apache 2.0. The required Python is at least 3.10 and below 3.14.

# the core alone, no engine
pip install outlines

# a local model through transformers, the only route to a full guarantee on a GPU
pip install "outlines[transformers]"

# llama.cpp and MLX on Apple Silicon
pip install "outlines[llamacpp]"
pip install "outlines[mlxlm]"

# note: this extra installs only the openai package,
# because Outlines talks to a vLLM server over its OpenAI-compatible API
pip install "outlines[vllm]"

# alternative automaton-building backends
pip install "outlines[xgrammar]"
pip install "outlines[llguidance]"

The outlines[vllm] extra is the most frequent source of surprise. In the package metadata it lists a single dependency, openai, because the vLLM integration goes through the OpenAI-compatible API. You have to install vLLM itself separately, and if you want the offline mode you need the vllm package in the environment anyway. Similarly outlines[sglang] brings in only openai, and outlines[tgi] only huggingface_hub.

The licence is a textbook case, and after a run of projects where the declaration disagrees with the contents that is a pleasant change. Three independent sources say the same thing. The license field on PyPI holds Apache-2.0, a proper SPDX identifier rather than the whole licence text pasted in. The LICENSE file on the main branch of the dottxt-ai/outlines repository answers with code 200 and contains the full Apache 2.0 text with the note Copyright 2023- The Outlines developers, while the LICENSE.md variant does not exist and returns 404. The published wheel outlines-1.3.3-py3-none-any.whl has a single file in outlines-1.3.3.dist-info/licenses/, namely LICENSE, again with the Apache 2.0 text, and it contains real code rather than metadata alone. The only thing missing is a License :: OSI Approved classifier in the PyPI classifier list, which holds only Programming Language :: Python :: 3. Tools that read classifiers instead of the license field may therefore show a blank.

Vendor risk deserves separate thought. The project is run by the company dottxt, the repository has about 15.7 thousand stars and is actively developed, but the same company sells a commercial API, available in the library as from_dottxt. The dottxt.ai site publishes no price list, the pricing subpage returns 404, and the only route is a contact form. If you plan to rely on the paid variant, you first have to talk to a salesperson, because the numbers are not public. The library itself under Apache 2.0 works without any of that, of course.

Beyond JSON Schema: regular expressions and grammars

JSON Schema support is table stakes and everyone has it, including Instructor and Pydantic AI. What sets Outlines apart is what it can do beyond schemas.

The outlines.types module exposes the functions regex, cfg and json_schema along with the classes Regex, CFG, Choice and JsonSchema. On top of that comes a small language for composing patterns: either, optional, exactly, at_least, at_most, between, one_or_more and zero_or_more. The outlines.types.locale submodule holds ready-made patterns, so far only American ones: locale.us.zip_code is \d{5}(?:-\d{4})?, and locale.us.phone_number is a number pattern in either the parenthesised or the hyphenated form. The airports and countries submodules exist but require the outlines[airports] and outlines[countries] extras, and without them they raise an ImportError with installation instructions.

Context-free grammars are written in Lark syntax. The outlines.grammars module ships two of them: grammars.json and grammars.arithmetic, loaded by the read_grammar function from a directory containing json.lark, arithmetic.lark and common.lark. You pass your own grammar as a string through cfg. This is the only route when you need to enforce a language that a JSON schema cannot describe: a SQL fragment, an expression in your own small configuration language, a robot command.

import outlines
from pydantic import BaseModel
from transformers import AutoModelForCausalLM, AutoTokenizer

class Invoice(BaseModel):
    number: str
    total: float
    currency: str

model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct"),
    AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct"),
)

# the generator builds the automaton once, so keep it outside the request loop
generator = outlines.Generator(model, Invoice, backend="outlines_core")

raw = generator("Write out invoice number FV/2026/114 for 1249.90 PLN")
invoice = Invoice.model_validate_json(raw)

Note the last line. The generator returns a string, not a Pydantic object. The SteerableGenerator.__call__ method calls self.model.generate(prompt, self.logits_processor, ...) and hands back whatever the model returned. Parsing into an object is on your side, and this is where Outlines differs from Instructor, which returns a ready instance.

from outlines.types import regex, either, cfg, json_schema, locale

# an invoice number in one specific format
invoice_no = regex(r"FV/\d{4}/\d{1,4}")

# a choice from a closed list, no guessing
priority = either("low", "medium", "high")

# a ready-made pattern from the locale submodule
postal_code = locale.us.zip_code

# a Lark grammar as a string
list_of_items = cfg(r"""
start: "[" item ("," item)* "]"
item: /[a-z]+/
""")

# a JSON schema given directly, with whitespace control
schema = json_schema({"type": "object", "properties": {"n": {"type": "integer"}}})

generator = outlines.Generator(model, invoice_no)

The python_types_to_terms function accepts a fair number of types directly: int, float, bool, str, a bare dict (translated into the grammars.json grammar), time, date, datetime, Enum classes, Literal, Union, list, tuple, a parameterised dict, Pydantic models and related structures, a schema builder from the genson library, and even a function signature. It does have a hard nesting limit: going past the tenth recursion level ends in a RecursionError. Deeply nested or recursive data models simply will not go through.

Outlines versus Instructor, doing the sums

The shortest possible statement of the difference is this: Instructor asks the model and retries when validation fails, so it works with any provider, but it pays for the retries and guarantees nothing. Outlines guarantees, but with steerable models it requires the weights inside your process. The rest is detail.

Let us do the sums for Instructor on an explicit assumption, since I have no production measurement: assume 1200 input tokens and 300 output tokens per request, and one retry needed in 5 percent of cases. Over a thousand requests that gives 1000 base calls plus 50 retries, so 1050 calls and 5 percent overhead on the request count. Input grows from 1,200,000 to 1,260,000 tokens, output from 300,000 to 315,000. That is an assumption, not a measurement, and with a difficult schema the error rate is often several times higher. The default max_retries=3 in Instructor's create method means at worst four paid calls for a single user request.

On the Outlines side with a steerable model there are no retries, because a non-conforming output cannot be produced. You pay for something else: GPU time held regardless of traffic, automaton compilation at startup, and the operational work around your own engine that with LiteLLM or Ollama you could skip. The break-even point therefore depends on load, not on the technology itself. At a few hundred requests per day a paid provider with retries comes out cheaper than a card in the cloud. With a steady, large stream the proportions flip.

There is also an Instructor advantage that is easy to forget when comparing guarantees. Pydantic validators can check meaning, not just shape: that a date lies in the future, that line items add up to the total, that an identifier exists in the database. An automaton derived from a schema cannot express any of that. If your problem is content rather than syntax, token masking will not solve it and it makes more sense to reach for Instructor or Guardrails AI.

Common mistakes

The first and most expensive is assuming that from_openai gives you the token-masking guarantee. It does not. You get the provider's response_format, exactly what you would get without Outlines, only through a different interface. Code that passes tests locally on from_transformers loses the property it was built on the moment you switch to OpenAI.

The second is silent degradation with a vLLM older than 0.12. The structured_outputs field is then ignored, the request finishes with a success code, and the output carries no constraints at all. Check the server version before concluding that enforcement works, and add a parsing assertion on your side as a safety net.

The third is building a Generator inside the request handler. Compiling an automaton from a complex schema can take a noticeable amount of time, and the class is designed so that the processor is built once and kept. In a web service, create the generators at process startup, one per schema.

The fourth is confusing syntactic correctness with truth. An enforced schema guarantees that the total field will be a number. It does not guarantee it is the right number from the invoice. A layer that checks meaning is still needed.

The fifth is copying code from tutorials written before mid-2025. Calls like models.transformers(...) and generate.json(...) disappeared in 1.1.0. If an example contains neither from_ in the loader function name nor the Generator constructor, it predates the break.

The sixth is installing outlines[vllm] expecting vLLM to show up. What shows up is openai. Check the extra's dependency list before you build a container image without an actual engine. Local models usually come from Hugging Face, which is a separate step and a separate cost in disk space.

FAQ

Does Outlines work with OpenAI and Anthropic?

It does, but in black box mode. The constraint is passed to the provider as response_format, and Outlines never sees the probability distribution and masks nothing. Regular expressions and grammars are unavailable there and end in a TypeError with a clear message.

When did version 1.0 ship and do old tutorials still work?

Version 1.0.0 appeared on PyPI on 18 June 2025. The deprecated interface was removed in 1.1.0 on 10 July 2025. Code using models.transformers or generate.json will not run on the current 1.3.3.

Does the guarantee mean the answer is factually correct?

No. The guarantee covers shape only: the output will always parse against the schema. Field contents can be invented, and you still need semantic validation or a check against a source of truth.

Which backend should I pick?

The defaults are sensible: outlines_core for JSON schemas and regular expressions, llguidance for grammars. Reach for xgrammar or llguidance when building the automaton from your schema turns out too slow, and compare timings on your own case, because the differences depend on the shape of the schema.

Does Outlines replace Instructor?

Not in every scenario. It replaces it where you hold the weights locally and care about a guarantee without retries. It does not replace it where you use a closed provider or need validators that check the meaning of the data rather than its shape.

How large a schema can Outlines handle?

The hard limit is the tenth recursion level in python_types_to_terms, past which you get a RecursionError. The soft limit is automaton compilation time, which grows with schema complexity. Measure it on your own schema before you deploy.

The project documentation lives at dottxt-ai.github.io/outlines, the source code in the dottxt-ai/outlines repository, and the releases and metadata on PyPI.