Task statement 6.3 · Prompt and context engineering

6.3 · Output Handling

Shelfwise, the course example, ends every answer with one line, such as "12 in stock at Leeds (feed 09:00)", which staff may repeat to a customer. Output handling decides what the service checks before that line, or any structured reply, is trusted.

Why it's tested

The tempting belief is that output which parses can be used. Structured outputs make JSON parse, but a reply cut off at its token limit still breaks, and a schema-valid record can hold a count nobody looked up. A fluent line can also state a figure nobody computed or point to a document nobody sent.

KnowledgeStructured output patterns

Structured output patterns are the three ways to get a reply your code can read. JSON outputs send a JSON schema in output_config.format. Constrained decoding, which allows only tokens that keep the reply valid, makes the text block match it. This suits a reply that is itself data, such as an extraction. Strict tool use, strict: true on a tool (2.4), guarantees a tool's name and input instead, so it suits data passed into a function. A prompted format (6.2) guarantees nothing but keeps the reply as text. Citations need that, because citations with output_config.format return a 400 error. Schema design belongs to 2.5.

More precise than the guide. The structured-outputs page calls JSON outputs "always valid", then lists exceptions (as of 2026-10-05): a reply stopped by max_tokens may be incomplete, a refusal may not match the schema, and an enum value may differ in capitalisation. Exam answer: structured outputs fix the shape of a finished reply only, so read stop_reason before parsing and compare enum values ignoring case.

KnowledgeResponse validation

Response validation is the code that decides whether a reply may be used, and it runs in a fixed order. It reads stop_reason first: for a final answer, end_turn means Claude finished, while max_tokens, model_context_window_exceeded and refusal mean the content is cut short or is not an answer. Next comes the schema. Last come the rules no schema expresses: ranges, since structured outputs don't support minimum and maximum; consistency between fields, such as an end date after its start; and references that must exist, such as a catalogue code.

When a check fails, a validation-retry loop helps only if the right value is in what Claude was sent:

  1. Write each failure as a sentence Claude can act on: "end_date 2026-03-01 is before start_date 2026-03-04".
  2. If the input holds the value, resend it with the rejected reply and those sentences, and validate the new reply.
  3. If it doesn't, keep the field null and flag the record, because a retry only invites an invented value.
  4. Cap the attempts, and send whatever still fails to a person.
  5. Don't resend a refused request to the same model: read stop_details, then fall back to another model, change the prompt, or show a fixed message. After max_tokens, raise the limit.

Interactive · where each kind of bad output is caught

Five checks your code runs on a reply, read from top to bottom, each passing the reply down to the next. One, stop_reason: end_turn goes on, while max_tokens or refusal stops the reply here; on failure, raise max_tokens, or for a refusal fall back to another model, change the prompt or show a fixed message, rather than resending the request to the same one. Two, parse with a JSON parser or a strict pattern, never eval; on failure, catch the error, keep the text, and retry or show a fixed message. Three, schema: types, required fields and enum values; on failure, retry with the errors named, and compare enum values ignoring case. Four, business rules: ranges, cross-field consistency and references that must exist; on failure, retry once if the input holds the value, otherwise set null and send it to a person. Five, grounding: claims against sources and numbers against code; on failure, remove the unsupported claim, and have a person review high-stakes output. A reply that passes all five is used. Your code, in order When the check fails 1 stop_reason end_turn GOES ON; max_tokens OR refusal STOPS passes 2 parse A JSON PARSER OR A STRICT PATTERN, NEVER eval passes 3 schema TYPES, REQUIRED FIELDS, enum VALUES passes 4 business rules RANGES, CROSS-FIELD, REFERENCES THAT MUST EXIST passes 5 grounding CLAIMS AGAINST SOURCES, NUMBERS AGAINST CODE passes use the output fails max_tokens: RAISE THE LIMIT refusal: ANOTHER MODEL, NOT A REPEAT fails CATCH THE ERROR, KEEP THE TEXT RETRY, OR A FIXED MESSAGE fails RETRY WITH THE ERRORS NAMED enum: COMPARE IGNORING CASE fails VALUE IN THE INPUT: ONE RETRY NOT IN IT: NULL AND A PERSON fails REMOVE THE UNSUPPORTED CLAIM HIGH STAKES: A PERSON REVIEWS

Choose a kind of output to see which check catches it.

Cut off at the limit: stop_reason is max_tokens, so the JSON may be unfinished. Don't parse it; raise max_tokens, or ask for less in each request.

Refusal: stop_reason is refusal, and the content may not match the schema. Read stop_details, then fall back to another model, change the prompt, or show a fixed message, rather than resending the request to the same model.

Sentence before the JSON: a prompted format with no schema behind it. Your code catches the parser's error, keeps the text for the log, and retries or shows a fixed message.

Enum in another case: structured outputs can return "Open Ticket" where the schema says "Open ticket", with no error. Compare enum values ignoring case, and avoid values that differ only in case.

End date before start: every field has the right type, so the schema passes. A cross-field rule in your code catches it, and because the input holds the right dates, one retry that quotes the failed check usually fixes it.

Count with no lookup: the line parses and breaks no rule, but no lookup in this turn holds that count. The grounding check catches it, and Shelfwise shows a fixed message instead.

Clean reply: it ended with end_turn, parses, matches the schema, passes the rules and agrees with its sources, so your code uses it.

Time reads downward. Blue marks a check in your code, ember the action when it fails, and green the only way to use.

KnowledgeDefensive parsing

Defensive parsing means reading Claude's output without assuming its shape. A prompted format may arrive with a sentence before the JSON, a code fence around it, or a field missing or extra, so use a real JSON parser, catch its error, default the optional fields and reject what still doesn't fit. Never call eval on the text, which RFC 8259 calls an unacceptable security risk, because the text could contain code. A tool's input arrives already parsed, so read it as given. In a stream (2.3), tool input comes as input_json_delta fragments, so join them and parse, guarded, when the block closes. Every parser needs a failure path that ends in a retry, a fixed message or a person, never a silent default.

KnowledgeScepticism toward confident output

Scepticism toward confident output, the guide's "skepticism toward confident output", means treating a fluent answer as a claim to check. Claude can state a wrong figure or cite a source that doesn't exist as smoothly as a right one, and the docs say their techniques reduce hallucinations without removing them. So give Claude fixed words for "I don't know" (6.2), limit it to the documents you send, and have it quote a source for each claim and withdraw any claim it cannot support. With citations, the API extracts each cited_text from your document, so every pointer is valid. Take figures from code, yours or the code execution tool, and have a person review high-stakes output.

SkillWorked example: a count with no lookup behind it

In the session incident (2.5), Leeds staff asking about 4 mm drill bits at 15:10 got "12 in stock", another chat's 09:40 count, with no lookup in that turn, shown like any other answer. After the session fix, the team added a check on each finished line. Replayed in today's answer format, the 15:10 reply fails it:

# 1 parse: the line must match one of two patterns, or it fails
<n> in stock at <branch> (feed HH:MM)  |  No record at <branch> (feed HH:MM)
# 2 ground: a stock_lookup result from this turn must hold that count and feed_time
"12 in stock at Leeds (feed 09:00)"   stock_lookup results this turn: none
widget: "Stock check failed, check the shelf"

The widget streams the line greyed as "checking", then marks it checked or replaces it when the message ends. "No record" passes only after a same-turn lookup found nothing, not after an error.

Before your code uses Claude's output, ask: did it finish, does it fit, and is it true? Not finished → act on stop_reason instead of parsing. Doesn't fit → retry once with the failed check named if the input holds the answer, otherwise null and a person. Not shown true → compare it with the lookup, source or computed figure.
Takeaway
  • JSON outputs and strict tool use fix the shape of a finished reply, not its truth.
  • A retry pays only when the input holds the right value and the retry quotes the check that failed.
  • Figures come from code, claims from a quote or citation, and high-stakes output goes to a person.

check yourself

0 / 3 answered

A research tool extracts every finding from long papers as JSON, with its schema in output_config.format, so the team expected every reply to parse. About 2% fail json.loads with "Unterminated string". All of them come from the longest papers, return HTTP 200, and report output_tokens of exactly 4,096, the request's max_tokens.What is the most likely root cause?

Select one.

A travel agency extracts flight bookings from confirmation emails with structured outputs, and every reply matches the schema. Yet 3% of records put arrival before departure or use an airport code missing from the agency's table, though the emails hold the right details. No bad record may reach the booking system, and staff can check only a few a day.Which change is the most proportionate response?

Select one.

A sales-analytics assistant writes a quarterly board summary from a 4,000-row CSV of orders placed in the prompt. The summary reads confidently, but an audit found two of six growth percentages wrong and an "industry benchmark of 12%" that appears in no file the team sent. The board makes budget decisions from these summaries.Which TWO changes most directly stop wrong figures and unsupported claims from reaching the board?

Select two.

sources

  1. Claude Platform Docs, "Structured outputs": JSON outputs in output_config.format and strict tool use as two features; constrained decoding and "always valid"; invalid outputs on refusal and max_tokens; enum capitalisation not guaranteed; numerical constraints not supported; citations incompatible with JSON outputs (400). Checked 2026-10-05.
  2. Claude Platform Docs, "Strict tool use": strict: true guarantees a valid tool name and schema-matching input. Checked 2026-10-05.
  3. Claude Platform Docs, "Stop reasons and fallback" and "Refusals and fallback": the stop_reason values and what to do for each; a refusal as a normal response with stop_details, usually served by another model. Checked 2026-10-05.
  4. Claude Platform Docs, "Handle tool calls": tool input as an object; instructive error messages that say what went wrong and what to try next. Checked 2026-10-05.
  5. Claude Platform Docs, "Fine-grained tool streaming": concatenating input_json_delta fragments and parsing at content_block_stop; the accumulated string may be invalid JSON, so guard the parse. Checked 2026-10-05.
  6. Claude Platform Docs, "Reduce hallucinations": allowing "I don't know"; direct quotes; verifying each claim with a supporting quote and retracting the rest; restricting Claude to the documents provided; validating critical information. Checked 2026-10-05.
  7. Claude Platform Docs, "Citations": cited_text extracted by the API, so citations are guaranteed to contain valid pointers. And "Code execution tool": calculations run as code in a sandbox. Checked 2026-10-05.
  8. IETF, RFC 8259 §12, "Security Considerations": parsing JSON with eval() "generally constitutes an unacceptable security risk". Checked 2026-10-05.
  9. The answer check, its two patterns and the "Stock check failed, check the shelf" message are this course's additions to Shelfwise; the 15:10 reply comes from 2.5 and the answer line from 5.1.