Definition
A program that reads 10,000 bank complaints needs each answer as clean data, {"category": "card", "urgent": true}, not a friendly paragraph. One stray sentence like "Sure! Here is the JSON:" can crash the whole pipeline.
Structured output means getting an LLM to return data in a fixed, machine-readable format, usually JSON that matches a schema listing the required fields, their types and allowed values. Techniques range from describing the format in the prompt with an example, through provider features such as JSON mode or schema-constrained generation that force valid output, to tool or function calling, where the model fills in the arguments of a defined function. Outputs should always be parsed and validated in code, with a retry or fallback when they fail.
In simple words
Instead of asking someone to tell you about a train journey, you hand them a form with boxes: From, To, Date, Class. Structured output is giving the model a form to fill instead of a blank page.
Overview
- Structured output means getting an LLM to reply in a machine-readable format such as JSON.
- Programs can then use the reply directly, for example to fill a database or call a function.
- A schema lists the fields, types and allowed values.
- Many APIs offer JSON mode or schema-constrained decoding that guarantees valid structure.
- Outputs should still be validated, because valid JSON can contain wrong values.
- Low temperature makes structured outputs more reliable.
Why it matters
Most real LLM applications feed model output into other software: databases, dashboards, forms or other programs. Structured outputs make LLMs dependable components rather than chatty text generators.
Key terms
- JSON
- A text format for data using objects {}, arrays [], strings, numbers, true, false and null.
- Schema
- A description of the required fields, their types and allowed values.
- Validation
- Checking in code that output matches the schema before using it.
- Enum
- A field restricted to a fixed list of allowed values.
- Function calling
- The model returns a function name and arguments that your code then executes.
- Constrained decoding
- Restricting token choices during generation so the output must follow a grammar or schema.
Types
There are several types of structured output and several ways to obtain them.
1. JSON objects
Replies as JSON with named fields.
- How it works: The prompt or API specifies keys and types; json.loads parses the reply into a Python dictionary.
- Example: {"name": "Asha", "city": "Pune", "intent": "refund"} extracted from an email.
- When to use it: Use it whenever another program will read the output. Ask for JSON only, with no extra text, and give the exact keys you expect so parsing does not fail.
2. Schema-constrained outputs
Outputs forced to match a JSON Schema or a typed model such as a Pydantic class.
- How it works: Constrained decoding allows only tokens that keep the output valid, so required fields, enums and types are respected.
- Example: A sentiment field restricted to "positive", "negative" or "neutral".
- When to use it: Use it in production pipelines where malformed output would break the system.
3. Function (tool) calling
The model choosing a function and returning its arguments as structured data.
- How it works: Developers describe tools with names and parameters; the model returns a call such as get_weather(city="Delhi"), the program runs it and sends back the result.
- Example: A travel assistant calling a train-search function with source, destination and date.
- When to use it: Use it to connect LLMs to databases, APIs and actions safely.
4. Tables, lists and other text formats
Markdown tables, CSV, YAML, numbered lists or XML tags.
- How it works: The prompt shows the exact format, ideally with an example; simpler formats are easier for the model to follow.
- Example: A comparison of three phones returned as a CSV with columns name, price and battery.
- When to use it: Use them for human-readable reports or simple data exchange.
5. Validation and repair
Checking structured output and fixing or retrying when it is wrong.
- How it works: Parse the output, validate it against the schema and business rules, and on failure retry with the error message or fall back safely.
- Example: Rejecting a date of 31 February and asking the model again.
- When to use it: Use it always: never trust structured output without checks.
Explained in detail
Designing the schema
Start by deciding exactly what your program needs. For complaint triage you might need category (one of card, loan, account, fraud, other), urgent (true or false), customer_name (string or null) and a summary (string of at most 30 words). Use enums for categories so the model cannot invent new ones, allow null for information that may be missing, and keep the schema as small as possible. Field names and short descriptions act as instructions, so make them clear. Ask for the evidence too, for example a quote from the complaint that supports the category, which helps reviewers check the result.
Ways to get structured output
| Method | Reliability | Notes |
|---|---|---|
| Format described in the prompt with an example | Good, not guaranteed | Works with any model; always validate |
| JSON mode | Always valid JSON, but fields may differ | Still check required fields and types |
| Schema-constrained output | Matches the schema | Supported by several providers and open-source libraries |
| Function or tool calling | High; arguments follow the declared parameters | Natural for actions such as "create_ticket" |
Validate, then trust
Even with constrained output, the content can be wrong: a valid JSON object may still put a fraud complaint in the loan category. So validation has two layers. Syntactic validation parses the JSON and checks fields, types and allowed values; libraries such as Pydantic in Python or JSON Schema validators do this. Semantic validation checks that values make sense: dates are not in the future, totals equal the sum of line items, quoted evidence actually appears in the source text. When validation fails, retry once with the error message added to the prompt, and if it fails again, send the item to a person. Log failures, because patterns in them show how to improve the prompt or schema.
Function calling is how LLMs take actions. You describe tools to the model, for example get_train_status(train_number) or create_ticket(category, summary), each with a name, a description and typed parameters. When a user's request needs a tool, the model replies with the tool name and arguments instead of text. Your code checks the arguments, runs the real function, and sends the result back so the model can write the final answer. This keeps control in your code: the model only suggests calls, and your program decides whether to execute them, which is essential for anything involving money, personal data or irreversible changes.
How it works
- List the exact fields your program needs, with types and allowed values.
- Describe the schema in the prompt or use the provider's schema or tool features.
- Set a low temperature for consistent extraction.
- Parse and validate both format and meaning in code.
- Retry with the error, then fall back to human review and log failures.
In code
import json
SCHEMA = {"category": {"card", "loan", "account", "fraud", "other"}, "urgent": bool, "summary": str}
def validate(raw, source_text):
try:
start, end = raw.index("{"), raw.rindex("}") + 1 # tolerate chatter around the JSON
data = json.loads(raw[start:end])
except ValueError as e:
return None, f"not valid JSON ({e.__class__.__name__})"
for field, rule in SCHEMA.items():
if field not in data:
return None, f"missing field '{field}'"
value = data[field]
if isinstance(rule, set) and value not in rule:
return None, f"'{value}' is not an allowed {field}"
if isinstance(rule, type) and not isinstance(value, rule):
return None, f"{field} should be {rule.__name__}"
if len(data["summary"].split()) > 30:
return None, "summary longer than 30 words"
if data["category"] == "fraud" and "otp" not in source_text.lower() and "unknown" not in source_text.lower():
return data, "valid, but flag for review: fraud label without fraud words"
return data, "valid"
complaint = "Someone asked for my OTP and then money was taken from my account"
outputs = [
'Sure! Here is the JSON: {"category": "fraud", "urgent": true, "summary": "OTP scam, money debited"}',
'{"category": "scam", "urgent": true, "summary": "OTP scam"}',
'{"category": "fraud", "urgent": "yes", "summary": "OTP scam"}',
'{"category": "fraud", "urgent": true',
]
for raw in outputs:
data, status = validate(raw, complaint)
print(f"{status:<40} <- {raw[:50]}")Output
valid <- Sure! Here is the JSON: {"category": "fraud", "urg
'scam' is not an allowed category <- {"category": "scam", "urgent": true, "summary": "O
urgent should be bool <- {"category": "fraud", "urgent": "yes", "summary":
not valid JSON (ValueError) <- {"category": "fraud", "urgent": trueTry it yourself
Reading 10,000 forms
A scholarship office receives 10,000 scanned application letters and needs a spreadsheet.
Valid against the schema?
Schema: category is one of card, loan, account; urgent is true or false.
{"category": "card", "urgent": "maybe"}
{"category": "card"}
{"category": "loan", "urgent": false}
{"category": "Loans", "urgent": false}
{"category": "account", "urgent": true}
Common mistakes
- Trusting that the output is valid JSON without parsing it.
- Allowing free-text categories instead of an enum.
- Checking format but never checking whether the values are correct.
Check yourself
Summary
- Structured outputs return data in a fixed schema, usually JSON, for other software to use.
- Use enums, nulls and small schemas; use schema features or function calling where available.
- Always validate format and meaning, retry once, then fall back to people.
Finished reading?
Sources: JSON Schema: getting started · Anthropic: prompt engineering overview