Task statement 5.1 · Model selection and optimization
5.1 · LLM Fundamentals
In Shelfwise, the course example, a Haiku-family model writes every stock answer one token at a time, and every token costs against the $0.01 ceiling. Three things explain an answer's cost and why the same question can come back worded differently: how tokens are produced and chosen, which model options exist, and how examples shape an answer.
share
Two reasonable instincts fail here. One sets temperature to 0 so that answers repeat exactly, but even temperature 0 is not fully deterministic, and the newest models reject a non-default temperature with a 400 error. The other adds one example to fix an answer's shape, and Claude copies that example's situation onto inputs it doesn't fit. Both treat the model as a lookup, when it chooses each token from several likely ones.
Start hereOne term this section adds
- sampling
- Choosing the next token from the probabilities the model gives each candidate. Temperature,
top_pandtop_kchange how that choice is made.
KnowledgeHow a response is generated
An LLM, a large language model, writes by next-token generation: it reads every token of the request, gives a probability to each token that could come next, picks one, appends it, and repeats until it finishes or reaches max_tokens. Tokenizers differ between models, and the newest models' tokenizer turns the same text into about 30% more tokens, so a count made on one model is wrong for another. The free count_tokens endpoint takes most of the same inputs as a Messages request and returns an estimate of its input tokens for the model you name, so Shelfwise's ≈7,000-token requests must be counted again after any model change.
The context window is everything the model can read and write in one request, the output and thinking included. It is 200,000 tokens on Haiku-family models and 1 million on the newest Opus- and Sonnet-family models, but more is not automatically better, because recall degrades as the context grows, which the docs call context rot. 2.1 treats the window as a hard limit in requirements.
Sampling settings change how each next token is picked. Temperature reshapes the model's probabilities: lower values favour the likeliest tokens and give more conservative output, and at the default of 1.0 rarer tokens keep their chance. top_p keeps the likeliest candidates whose probabilities add up to that share, and top_k keeps the k likeliest. Non-determinism means identical requests can return different text, even at temperature 0, so you design for it: check what an answer must contain, and grade evals on properties, not exact wording. On the newest models all three settings are deprecated: a non-default temperature, a top_p below 0.99 or any top_k returns a 400, and the docs say to steer by prompting instead.
Interactive · how a sampling setting changes the choice of the next token
Choose a sampling setting for the next token of a stock answer.
Temperature 1.0, the default: the sampler draws from the probabilities as the model gave them, so "stock" comes out most of the time and "Leeds", "total" or "the" now and then.
Temperature near 0: the most likely token takes almost all the probability, so answers become more conservative and more alike. The docs still say results are not fully deterministic, and the newest models reject any non-default temperature with a 400.
top_p 0.9: the sampler keeps the most likely tokens until their probabilities add up to 0.9, here "stock" and "Leeds", and picks between those two. The newest models reject a top_p below 0.99 with a 400.
KnowledgeModel options: thinking, effort and fast mode
Extended thinking and adaptive thinking let Claude reason in thinking blocks, billed as output, before it answers. Extended thinking is the manual mode: you set thinking: {type: "enabled", budget_tokens: N}, with N at least 1,024 and below max_tokens, and Claude thinks against that target. Adaptive thinking is Claude deciding per request whether and how deeply to think. On the newest models it is the only thinking mode, so budget_tokens returns a 400 (2.3). On the latest generation of those models, adaptive thinking is also on by default. Haiku-family models offer only extended thinking, and Shelfwise sets thinking to disabled, because a stock lookup needs no long reasoning and thinking tokens would count against the $0.01 ceiling.
Effort is the setting, output_config.effort, that tells Claude how much work to put into a response, on every family except Haiku. Its levels are low, medium, high, xhigh and max, with high the default on most models. It shapes every output token, thinking included, so lowering it is the first move when a thinking workload costs too much or runs too slowly. Effort is a behavioural signal, though: Claude still thinks on hard problems at low effort, and only max_tokens is a strict ceiling on output.
Fast mode runs the same Opus-family model with a faster inference configuration: up to 2.5 times the output tokens per second, with no change in intelligence, at a premium price per token. You opt in with speed: "fast" and a beta header. It speeds up output, not the first token, it isn't available in the Message Batches API, and switching speed misses the prompt cache.
More precise than the guide. The guide lists fast mode as a model option with no conditions. It is a research preview for the newest Opus-family models, granted on request on the Claude API and not offered on Amazon Bedrock or Google Cloud (as of 2026-10-04). Exam answer: fast mode is an option for the newest Opus-family models on the Claude API, not a setting every model or platform accepts.
SkillPrompting with zero, one or several examples
Zero-shot, single-shot and multi-shot prompting differ in how many worked examples the prompt carries. Zero-shot is instructions alone, which suits tasks whose exact output shape doesn't matter. Single-shot adds one example, which fixes the format quickly, but Claude tends to copy everything about it, including the situation it happens to show. Multi-shot adds several, and the docs call this technique few-shot (multishot): they suggest 3–5 examples that are relevant, diverse enough to cover edge cases, and wrapped in <example> tags so Claude can tell them from instructions. Examples steer format and judgement more reliably than sampling settings, and they keep working when the model changes.
Shelfwise's stock answers went through all three. Staff wanted one line with the count, the branch and the feed's time:
# zero-shot: the line described in the stock rules, no example # single-shot: one example, "12 in stock at Leeds (feed 09:00)" "brass hinge 80 mm" (not in the feed) → "0 in stock at Leeds (feed 09:00)" # multi-shot: three examples: 12 in stock, 0 in stock, no record "brass hinge 80 mm" → "No record at Leeds (feed 09:00)"
The single example showed only a product with stock, so Claude gave a count for a product the feed didn't hold, breaking the rule to say "no record" instead of guessing. Three examples that differ in exactly the cases that matter fixed it, and because they sit in the stock rules inside the cached ≈6,000-token prefix, each request reads them at the cache price. A lower temperature could not have shown Claude the missing case, and the Agent SDK's options have no temperature setting anyway.
max_tokens for a hard ceiling, fast mode for Opus-family output speed.- Claude picks each token by sampling, so identical requests can differ even at temperature 0, and the newest models accept only the default sampling settings.
- Token counts depend on the model's tokenizer, so recount a prompt with
count_tokensbefore moving it to another model. - Extended thinking takes a
budget_tokensyou set; adaptive thinking lets Claude decide, steered by effort; fast mode buys Opus-family output speed at a premium.