Task statement 5.3 · Model selection and optimization

5.3 · Model Selection and Tradeoffs

In Shelfwise, the course example, the model family decides whether an answer can stay under $0.01: with the prefix cached, Haiku-family pricing lands near $0.007 an answer and Sonnet-family pricing near $0.014. Choosing a family weighs quality, latency and cost, and a new model release has to be checked for what it breaks before staff see it.

Why it's tested

Two reasonable instincts fail when choosing or changing a model. One picks from the price list, the strongest to be safe or the cheapest per token, when only your own evals show which one meets the requirement. The other changes only the model ID for a newer release, but a release can reject request settings the old model accepted, or change a default with no error.

KnowledgeOpus, Sonnet and Haiku

Claude's current models come in families that trade capability against latency and price per token. Of the three the guide names, Opus is the most capable and the slowest, Haiku the fastest and lowest-priced, and Sonnet sits between them. Context window and thinking support also differ, so a requirement can rule a family out before quality is measured. Thinking support is the clearest case (5.1 lists the modes and defaults). Haiku-family models offer only extended thinking, off unless you enable it. So among current models, a workload that needs thinking off fits Haiku-family. The newest Sonnet-family model can turn off only its up-front thinking (thinking before the reply) through the API, and the newest Opus-family model can't turn it off at all.

Opusmost capable

Long-running agentic coding, complex reasoning, large refactors and computer use, at moderate latency.

LimitsA 1M-token window and 128k output, and adaptive thinking that can't be turned off.

Sonnetbalance

Code generation, data analysis, content creation and agentic tool use, with fast responses.

LimitsThe same window and output, with adaptive thinking on by default and effort available.

Haikufastest

Real-time features, high-volume processing and subagent tasks, at the lowest latency and price per token.

LimitsA 200k-token window and 64k output, with extended thinking only and no effort parameter.

More precise than the guide. The guide compares Opus, Sonnet and Haiku. The models overview also lists a fourth current family, Fable, above Opus for demanding reasoning and long-horizon agentic work (Mythos-family models are invitation-only), and says that if you are unsure, start most workloads on Opus (as of 2026-10-04). Exam answer: among the three named families, Opus is the capability-first choice and the other two give up capability for speed and a lower price; Fable, where it appears, ranks above Opus.

KnowledgeTrading quality, latency and cost

The docs' guide to choosing a model offers two starts. Efficiency-first begins on Haiku-family and moves up a tier only for a gap your tests show, and capability-first begins on Opus-family and moves down once quality is secured. Before changing family, test each effort level on the current model, because effort trades quality against latency and cost inside one model, and fast mode buys Opus-family output speed at a higher price per token (5.1). Compare candidates on cost per completed task on your own traffic, hardest tasks included, because a model that costs more per token can finish in fewer turns. When traffic mixes routine and hard steps, combine models: in the advisor strategy a lower-cost model runs the loop and escalates hard decisions to a stronger one, and in the orchestrator strategy a stronger model hands independent pieces to lower-cost workers. 5.4 covers the mechanics behind each bill.

KnowledgeBreaking changes across model releases

A breaking change is a difference in a model release that makes working code fail or behave differently. Three groups of settings return a 400 error. The newest models reject non-default sampling settings (5.1) and a manual budget_tokens (2.3). Claude 4.6 and later models also reject assistant prefill, a final assistant turn that starts Claude's reply. The newest Opus-, Sonnet-, Fable- and Mythos-family models reject a forced tool call: tool_choice, the field that controls tool use, set to any or tool. The migration guides replace a format prefill with structured outputs, and forced tool choice with auto plus strict tool use. Other changes raise no error: the newer tokenizer counts the same text as more tokens (5.1), the newest Opus-family model runs at a lower default effort than its predecessor, and the latest models write more concisely and follow instructions more literally. So read the target model's migration guide, then change the request on a branch, re-run the evals, recount tokens and pin the new ID in configuration (2.6), with the platform's own ID and dates on Bedrock or Google Cloud (2.3).

SkillChecking a candidate model for Shelfwise

The deprecations page lists a single active Haiku-family model, with a tentative retirement date, so Shelfwise's team tried the next tier up on a branch, changing only the model ID in config/release.toml:

# branch: model = newest Sonnet-family ID; options still say thinking "disabled"
eval run → 150/150, no errors
usage → thinking billed as output
count_tokens → ≈9,100 input tokens a request, not ≈7,000
# Claude Code can't turn thinking off on this model; here "disabled" was ignored
cost per answer → over $0.01 (≈$0.014 before the extra tokens)

Nothing failed, which makes this change dangerous. Claude Code, which the Agent SDK runs, can't turn thinking off on this model, and in this trial the SDK's "disabled" was ignored rather than rejected. Quality held, but cost ruled the candidate out, because Sonnet-family pricing already lands near $0.014 an answer before thinking and a tokenizer that adds about 30%. The team closed the pull request with the result recorded: any replacement must be priced near Haiku-family rates, or the $0.01 requirement goes back to the owners.

Interactive · the checks a candidate model passes before Shelfwise switches

Four checks read from top to bottom. One: the API accepts the request. Two: the eval set of 150 questions. Three: tokens and cost per answer against the 0.01 dollar ceiling. Four: the decision in the release configuration. In the default state the current Haiku-family model passes all four: the request is accepted, 150 of 150 pass, about 7,000 tokens a request and about 0.007 dollars an answer, and the pinned ID stays. 1 · API accepts request VALIDATION accepted accepted, no error THINKING OFF, AS CONFIGURED THINKING STAYS ON, SILENTLY if accepted 2 · Eval, 150 questions QUALITY 150/150 STOCK-RULES-V13 QUALITY HOLDS if quality holds 3 · Tokens and cost COST PER ANSWER UNDER $0.01? ≈7,000 tokens a request ≈9,100 tokens a request ≈$0.007 AN ANSWER PLUS THINKING: OVER $0.01 if under the ceiling 4 · Decision CONFIG/RELEASE.TOML keep the pinned ID pull request closed NO CHANGE HAIKU-FAMILY ID STAYS

Choose the model behind Shelfwise's request and follow it through the checks.

The current Haiku-family model: thinking stays off as configured, the eval set passes, and with the prefix cached an answer costs about $0.007, so the pinned ID stays.

The newest Sonnet-family model with the same options: the request is accepted, but Claude Code can't turn thinking off on this model, and in this trial "disabled" was ignored, so thinking is billed as output. Quality holds; the extra tokens and Sonnet-family pricing put every answer over $0.01.

Time runs downward: green passes, ember fails, and in this trial the ignored setting raised no error.
Ask one question: which requirement binds first, quality or latency and cost? Quality → sweep effort first, then move up a family or add a stronger advisor for the hard steps. Latency and cost → the smallest family that passes your evals.
Takeaway
  • Opus, Sonnet and Haiku trade capability against latency and price per token, and context window and thinking support can rule a family out first.
  • Price candidates per completed task on your own traffic, not per token.
  • A model release can reject old request settings with a 400 or change a default silently, so read its migration guide before the model ID changes.

check yourself

0 / 3 answered

A code-review bot ran on the previous Opus-family model with no effort field, and only its model ID was changed to the newest Opus-family model. Requests succeed and visible comments are as long as before, but output tokens per review fell by about a third, and it catches 39 of 50 seeded multi-file bugs, down from 46.What is the most likely root cause?

Select one.

An insurer's claims agent runs on a Sonnet-family model. Most turns are routine lookups and form filling, each step depending on the last, but its few coverage decisions per claim are wrong too often, even at the highest effort level. An Opus-family model running the whole agent gets them right but costs several times as much per claim.Which change best fixes the coverage decisions while keeping most tokens billed at the lower rate?

Select one.

A ticket-routing service on an earlier Sonnet-family model that accepted both sends two kinds of request. One forces its route_ticket tool with tool_choice: {"type": "tool"}. The other ends messages with an assistant turn holding {, so the reply starts as JSON. On a branch that targets the newest Sonnet-family model, both requests return 400.Which TWO changes make both requests valid on the new model and keep each reply checked against a schema?

Select two.

sources

  1. Claude Platform Docs, "Models overview": each family's description and comparative latency; context window, maximum output, thinking mode and default effort per model; every model ID a pinned snapshot; the fourth family, Fable. Checked 2026-10-04.
  2. Claude Platform Docs, "Choosing the right model": capabilities, speed and cost as criteria; effort often a better lever than switching models; efficiency-first and capability-first starts; the selection matrix of use cases; combining models. Checked 2026-10-04.
  3. Claude Platform Docs, "Optimizing for cost and intelligence": compare models on cost per completed task, including the hardest tasks; sweep effort first; the advisor and orchestrator strategies. Checked 2026-10-04.
  4. Claude Platform Docs, "Thinking": thinking modes per model; Haiku-family extended only with thinking off by default; disabled rejected on the newest Opus-family model; between_tools on the newest Sonnet-family model. And Claude Code Docs, "Model configuration": thinking can't be turned off on the newest Opus and Sonnet models, and an off setting has no effect there. Checked 2026-10-04.
  5. Claude Platform Docs, "Migrating to Claude Opus 5.5" and "Migrating to Claude Sonnet 5.5": settings that return a 400 (thinking, sampling, prefill, forced tool choice) and their replacements; about 30% more tokens than Haiku-family for the same text; concise, explicit-direction behaviour. And "What's new in Claude Opus 5.5": thinking can't be disabled; the lower default effort. Checked 2026-10-04.
  6. Claude Platform Docs, "Model deprecations": the active Haiku-family model and its tentative retirement date; test replacements well before retirement. And "Release notes": the newest Opus- and Sonnet-family launches and their breaking changes. Checked 2026-10-04.
  7. Claude Platform Docs, "Prompting best practices": the latest models are less verbose and trained for precise instruction following. Checked 2026-10-04.
  8. Shelfwise's Sonnet-family trial, its eval result, the thinking billed despite the setting and the ≈9,100-token count are this course's example details; the ≈$0.007 and ≈$0.014 figures come from the 2.1 worked example and are recomputed from the pricing page before each publish.