Task statement 5.4 · Model selection and optimization
5.4 · Cost and Token Management
In Shelfwise, the course example, the $0.01 ceiling is a cost model: about three requests an answer, each carrying ≈7,000 input tokens of which ≈6,000 are a cached prefix, at Haiku-family prices. The team must budget those tokens, read what each request used, and price it in a form a week of usage logs can check.
share
Two reasonable shortcuts give the wrong cost. One prices a request from its input size alone, but output costs five times as much per token, and thinking is billed as output. The other caches every prompt on the theory that caching always saves money, but a write costs more than plain input, so a prefix that is never read before it expires costs more cached than uncached.
KnowledgeBudgeting tokens per request
A token budget has an output side and an input side. max_tokens caps what one response writes, thinking included (thinking tokens count toward it and are billed as output, 2.3). A response that reaches the cap stops with stop_reason: "max_tokens", so the cap bounds cost but can cut an answer off. Effort is the softer lever where a model supports it, because lower effort uses fewer output tokens (5.1). Input is the system prompt, the tool definitions plus a tool-use system prompt the API adds, the history and any retrieved context. The API keeps no state between requests, so the history is resent and grows every turn. Shelfwise budgets ≈7,000 input tokens a request, starts a new session for each staff chat (2.5), and turns thinking off, because a Haiku-family model has no effort parameter to lower.
KnowledgeToken usage tracking
Every response carries a usage object for billing. input_tokens counts only the input after the last cache breakpoint, and with cache_read_input_tokens and cache_creation_input_tokens it makes up the request's total input. The cache_creation object splits writes into ephemeral_5m_input_tokens and ephemeral_1h_input_tokens, and output_tokens_details.thinking_tokens says how much of output_tokens was thinking. Token usage tracking means logging these fields for every request, tagged with the feature that sent it. The tag is needed because the Usage and Cost Admin API (2.2) reports by API key, workspace, model and service tier, so it sees a feature only if that feature has its own key. A workspace is the cost boundary above a key, with its own monthly spend limit, alerts and per-minute rate limits.
KnowledgeA cost model in multipliers
A cost model prices each usage field as a multiple of the model's base input price. On current models output costs five times the input price. A five-minute cache write costs 1.25 times it and a one-hour write twice, while a read costs a tenth on most models and less on the newest Opus-, Fable- and Mythos-family models. The Message Batches API halves input and output, stacking with the caching multipliers (2.3). The pricing page's "Cost optimization strategies" add the right model (5.3) and monitoring usage. The grid maps each line of the cost model to the field that counts it.
input_tokens: everything after the last breakpoint.
ephemeral_5m_input_tokens and ephemeral_1h_input_tokens, by lifetime.
cache_read_input_tokens: the prefix reused from the cache.
output_tokens, thinking included.
KnowledgeWhen caching pays off
Prompt caching saves money only when reads repay the write premium. Against sending the prefix uncached, a five-minute write pays for itself after one read (1.25 + 0.1 is less than 2) and a one-hour write after two (2 + 0.2 is less than 3). Because each read refreshes the lifetime (2.3), the five-minute lifetime suits requests less than five minutes apart and the one-hour lifetime suits gaps of up to an hour. With longer gaps every request writes and none reads. A prefix below the model's minimum cacheable length, 4,096 tokens on the current Haiku-family model, is processed uncached with no error, and both cache fields read 0. The hit rate is cache-read tokens as a share of all input tokens, as Claude Code's hit_ratio reports it. Cache check-pointing, which the docs call placing cache_control breakpoints (2.3), decides which tokens the reads cover.
SkillPricing a Shelfwise answer from its usage
Shelfwise's first cost log priced each request as input_tokens at the input price plus output_tokens at five times it. A week of logs averaged about $0.005 an answer, yet the logged total came to only about three-quarters of the Cost API's daily figure for Shelfwise's workspace. Costs below are in units of one token at the base input price:
# one warm request, from its AssistantMessage usage input_tokens 1,000 · cache_read_input_tokens 6,000 · cache_creation_input_tokens 0 # v1 log, per answer: 3 requests, ≈400 output tokens 3 × 1,000 + 5 × 400 = 5,000 units → ≈$0.005, under-counted # cause: input_tokens stops at the breakpoint, so 18,000 cache reads went unpriced 3 × (1,000 + 0.1 × 6,000) + 5 × 400 = 6,800 units → ≈$0.007, agrees with the Cost API
With caching on, input_tokens stops at the breakpoint, so every answer's cache reads went unpriced. The fixed log prices all four lines and tags each request stock_answer or eval, so the eval runner's 150 questions count apart from staff answers. It also records the hit rate, about 0.86 at the Saturday peak, the most a 6,000-of-7,000 prefix allows. The first answer after more than five quiet minutes writes the prefix instead: (1.25 × 6,000 + 1,000) + 2 × 1,600 + 2,000 = 13,700 units, about $0.014, which is why 2.1 checks the week's mean cost per answer.
Interactive · what one Shelfwise answer costs, line by line
Choose a way of pricing the same answer and compare it with the $0.01 ceiling.
No caching: three requests of ≈7,000 full-price input tokens plus ≈400 output tokens at five times the price come to 23,000 units, about $0.023, more than twice the ceiling.
Cached, priced by the first log: input_tokens holds only the ≈1,000 tokens after the breakpoint, so the 18,000 cache-read tokens (1,800 units) are missing and the answer looks like $0.005.
Cached, every field priced: 1,800 units of cache reads, 3,000 of uncached input and 2,000 of output make 6,800 units, about $0.007, under the $0.01 ceiling.
cache_control at the default lifetime. Yes, but with gaps of five minutes to an hour → the one-hour TTL. No → leave it uncached, because nothing repays the write premium.- With caching on,
input_tokensis only the input after the last breakpoint, so total input adds cache reads and writes. - Turn usage into cost field by field, each with its own multiple of the input price: reads, writes, uncached input and output.
- Bound output with
max_tokensand effort, and bound input per conversation, because resent history grows every turn.