Task statement 4.3 · Prompt engineering and structured output

Enforce structured output using tool use and JSON schemas

Invoices, contracts and support emails pass through one extraction step before they load into the ledger. A tool with a strict JSON schema stopped the weekly crashes on malformed JSON, but a month later finance found due dates in the ledger that appear in no document.

Why it's tested

The tempting answers treat "the JSON is valid" and "the values are right" as one guarantee. A strict tool schema gives you the first, so the problem looks solved, but a record whose line items do not add up still loads. The opposite instinct fails too, because marking every field required looks rigorous but forces the model to write a value the document never contained.

KnowledgeHow a tool call returns a record

Claude answers an extraction request with a tool_use block whose input is the record your extraction tool's input_schema describes, so your code reads fields instead of parsing prose. Setting strict: true on the tool turns on grammar-constrained sampling, which means Claude can only generate tokens that keep the input valid against the schema. Syntax errors, wrong types and missing required fields are therefore not produced at all, rather than caught when a parser throws. Nothing here checks that a value is true, so relying on the schema to catch a wrong total works at the wrong layer.

  1. Define extract_invoice with the record as its input_schema and strict: true.
  2. Send the invoice text, the tool, and a tool_choice that requires a call.
  3. Claude replies with stop_reason: "tool_use" and a tool_use block.
  4. Check stop_reason, then read the block's input as the record.
  5. Validate the values, such as the line-item sum (4.4).
// the response your code receives
{ "stop_reason": "tool_use",
  "content": [{ "type": "tool_use", "name": "extract_invoice",
    "input": { "invoice_number": "INV-20417", "stated_total": 1410.00,
               "due_date": null, "category": "freight" } }] }

Interactive · where each error stops

Five kinds of extraction error travel down from Claude's output through generation, parsing and the downstream ledger; each output method stops a different subset of them malformed JSON unclosed brace wrong type "412" not 412 missing field no invoice_number invented value due_date guessed wrong meaning lines ≠ total ① GENERATION strict acts here ② YOUR CODE JSON.parse(output) ③ LEDGER trusted from here on parse throws blocked by strict reaches ledger blocked by strict reaches ledger blocked by strict reaches ledger invented to fill null now possible reaches ledger Prompting for JSON: syntax errors are caught only when the parser throws. Strict schema: shape errors stop at generation, unless the output is cut off or refused. due_date optional and nullable: null is a valid answer, so invention is less likely, not impossible. A string where a number belongs, or a missing field, parses cleanly and reaches the ledger. But due_date is required, so a document without one gets an invented date. A wrong sum is semantic. No schema checks it, so it needs your validator (4.4). BLOCKED AT GENERATION CAUGHT, RUN FAILS LESS LIKELY, NOT BLOCKED TRUSTED DOWNSTREAM

Prompt asks for JSON. Usually well-formed, never guaranteed. One error class is caught, by crashing the run; four reach the ledger.

Tool with a strict schema. Syntax, type and missing-field errors stop at generation. The required due_date now pushes the model to invent a value.

Strict schema, nullable due_date. Shape errors stop at generation, an invented date becomes less likely, and a wrong line-item sum still reaches the ledger.

Time reads downward. A strict schema stops the three shape errors at generation, unless the response is cut off at max_tokens or refused. Allowing null makes an invented value less likely, and only your validator catches a wrong sum.

More precise than the guide. The guide presents tool use with a JSON schema as guaranteed schema-compliant output. In the docs the guarantee comes from strict: true on the tool, or from JSON outputs set with output_config.format. Even strict output may not match the schema when the response ends with stop_reason: "max_tokens" (cut off) or "refusal", so check stop_reason before trusting input (checked 2026-10-01). Exam answer: tool use with a JSON schema eliminates JSON syntax errors.

Knowledgetool_choice: may, must, or must call this one

tool_choice decides, per request, whether Claude may or must call a tool. "auto", the default when tools are present, lets the model answer in text, so an extraction can come back as a prose summary with no record. "any" requires a call to one of the tools but lets the model choose which, which fits several extraction schemas and documents of unknown type. Forced selection, {"type": "tool", "name": "extract_metadata"}, requires that one named tool, so it fits a step that must run first. A prompt line saying "always call the tool" on top of "auto" only asks for what "any" guarantees.

// unlabelled intake
tools: [extract_invoice, extract_contract, extract_support_email]
tool_choice: { "type": "any" }
supplier contract PDF  → tool_use extract_contract

SkillMetadata first, then enrichment

In the guide's example, extract_metadata must run before the enrichment steps, which are extractions that depend on knowing what the document is. Turn 1 forces extract_metadata, so the response is always a tool_use block with the document type, date and parties. Your code reads doc_type and picks the enrichment tool. Turn 2 appends the turn-1 tool_use and its tool_result, then forces that tool. Without the forced first call, the model could enrich a document it never classified.

// turn 1: tool_choice { "type": "tool", "name": "extract_metadata" }
→ extract_metadata { "doc_type": "contract", "date": "2026-03-01",
                     "parties": ["Norhaven Freight", "Acme Ltd"] }
// your code: doc_type "contract" → enrichment tool extract_contract_terms
// turn 2: + tool_use, tool_result; tool_choice { "type": "tool", "name": "extract_contract_terms" }
→ extract_contract_terms { "auto_renews": true, "renewal_notice_days": 60 }

More precise than the guide. "any" and forced selection are not supported everywhere. On the newest Claude models (the current Opus, Sonnet, Fable and Mythos releases), they return a 400 error, and with manual extended thinking they fail on every model, so the docs substitute "auto" with strict tools, or structured outputs for a fixed JSON shape (checked 2026-10-01). Exam answer: tool_choice: "any" or a forced tool guarantees a structured call.

SkillA schema for documents that are not always complete

Nullable means a field's type allows null. A field that a legitimate document may lack should be nullable and left out of required. An invoice marked "payable on receipt" then has an honest answer for due_date, where a required string pushed the model to invent one. Nullable makes null available without forcing it, so invention becomes less likely, not impossible. Keep required for fields every valid document has, because a missing invoice number means the input is not an invoice.

Required

due_date must be present

"due_date": { "type": "string" }
"required": [..., "due_date"]

Result: the invoice gives no date, so the model writes the most plausible one, 30 days after issue: "2026-04-02". It is schema-valid and wrong.

Optional, nullable

due_date may be null

"due_date": { "anyOf": [
  { "type": "string" },
  { "type": "null" } ] }
// not listed in required

Result: the same invoice returns null, and the ledger records "no due date", which is what the document says.

Enums need the same escape routes. Add "unclear" for a document that fits two categories, so the model is not pushed into the nearer wrong one. Add "other" with a detail string for categories you have not listed, so a new kind of charge arrives as countable data, not a misfiled record. Strict output does not guarantee enum capitalisation, so compare values case-insensitively.

"category": { "enum": ["freight", "software", "services", "other", "unclear"] },
"category_detail": { "anyOf": [{ "type": "string" }, { "type": "null" }] }
// "Customs brokerage, March"     → "other", "customs brokerage"
// "Freight and on-site install"  → "unclear", null

A strict schema fixes the type, not how to read the source. 1.234,56 from a German supplier and 1,234.56 from a US one must both become 1234.56, and only the prompt can say so. Put format normalisation rules in the prompt alongside the schema: dates as ISO 8601 ("3rd March 2026" → 2026-03-03), amounts with a dot decimal and no separators, and currency as a three-letter code.

No JSON Schema keyword says "these line items must sum to that total", and strict schemas do not support minimum or maximum. Lines summing to 1,140.00 under a stated_total of 1,410.00 therefore pass, as does a value in the wrong field, such as the supplier as customer. These semantic errors belong to your validation code and its retry loop (4.4).

For each field, ask: can a legitimate source document lack this value? Yes → make it optional and nullable, so absence is an answer. No → keep it required, because its absence means the input is not what you think it is.
Takeaway
  • A strict tool schema removes syntax errors, except on a truncated or refused response. It never checks that the values are right.
  • "any" when several schemas exist and the document type is unknown. A forced tool when one extraction must run before enrichment.
  • "other" plus a detail field keeps a short category list extensible, and "unclear" gives an ambiguous document an honest answer.

check yourself

0 / 3 answered

The support-email extractor files each message under request_type, a strict enum of refund, shipping and technical. Data-deletion requests have started arriving and land in technical. Product wants to see which new request types appear, in the senders' words, before deciding what to add. The routing dashboard groups tickets by request_type.Which schema change best serves that goal?

Select one.

The contract extractor uses a strict tool. termination_notice_days is a required integer, and the 30% of contracts with no termination clause load with 30. Separately, EU suppliers write fees as 12.500,00, and some load as 12.5. The ledger stores fees as numbers, and finance must tell "no clause" apart from a real 30-day notice.Which TWO changes most directly fix both problems?

Select two.

The support-email pipeline offers classify_email, extract_refund_request and extract_address_change with tool_choice: "any". About one email in ten skips classify_email and goes straight to a specialist tool. Your code must look up the customer's account from the classification before the specialist extraction runs, because that prompt includes the account history.Which change most directly guarantees that classification runs first?

Select one.

sources

  1. Claude Platform Docs, "Structured outputs": JSON outputs via output_config.format; the "Invalid outputs" cases (stop_reason "refusal" and "max_tokens") and enum capitalisation; supported features (anyOf, enum) and unsupported ones (minimum, maximum). Checked 2026-10-01.
  2. Claude Platform Docs, "Strict tool use": strict: true as grammar-constrained sampling on the tool definition, and the "2"-instead-of-2 failure it prevents. Checked 2026-10-01.
  3. Claude Platform Docs, "Define tools", "Forcing tool use": the four tool_choice options, and the models and settings on which any and tool fail, with the docs' substitutes. Checked 2026-10-01.
  4. The extract_metadata forced-selection example, "other" + detail and "unclear" enum values, format normalisation rules, and the syntax-versus-semantic distinction: Claude Certified Architect — Foundations Exam Guide v1.0, Domain 4.3 and Appendix.