sample questions

10 of the 79 questions, spread across the domains. Open one to see the answer and why.

  1. A city council's planning office wants Claude to answer residents' questions about its 40-page planning guidance, which fits easily in one request, quoting the paragraph each answer relies on. A developer proposes a coordinator agent with zoning, heritage and fees subagents. No design has been tried yet.

    Which design is the most proportionate place to start?

    1. A coordinator agent that hands each question to a zoning, heritage or fees subagent holding only that topic's chapters.
    2. A routing workflow that classifies each question by topic and sends it to a prompt written for zoning, heritage or fees.
    3. One Claude call per question carrying the whole guidance and asking for the quoted paragraph, checked on a small eval set.
    4. One agent with a search tool over the guidance, which decides for each question how many searches to run before answering.
    Show the answer

    Answer: C.

    Correct: one call with the whole guidance, then measure. The guidance fits in one request, so one call is the simplest workable design, and a small eval set shows whether it falls short before steps are added. Why not the others: a routing workflow would also work and stays predictable, but it adds a classifier and three prompts before anything shows one prompt is not enough, against "most proportionate place to start"; one agent with a search tool adds a loop of requests to find text the call already carries; a coordinator with topic subagents costs the most and splits into chapters a document that never needed splitting.

  2. An insurer runs two Claude workloads. Claimants upload photos through a web form and wait on the page for a damage summary. Separately, 60,000 archived claims must be re-categorised for an audit due in two weeks. The finance lead wants the lowest total cost without making claimants wait any longer.

    Which design best meets both requirements?

    1. Keep the upload form on the real-time Messages API, and send the 60,000 archived claims through the Message Batches API.
    2. Run both workloads on the real-time Messages API, adding prompt caching to the shared instructions to cut input costs.
    3. Move both workloads to the Message Batches API, polling each batch every minute so that claimants see results quickly.
    4. Keep both workloads on the real-time Messages API, and lower max_tokens on the archive requests to reduce what they cost.
    Show the answer

    Answer: A.

    Correct: real-time form, batched archive. Claimants are waiting, so the form needs answers in seconds, while the archive has two weeks of slack, which easily absorbs the batch's 24-hour promise at half the price. Why not the others: caching on real time for both would work and does lower input costs, but the archive still pays full price, so it misses "lowest total cost"; batching both with frequent polling cannot make a batch finish sooner, so claimants could wait up to 24 hours; lower max_tokens trims each answer but leaves the price per token unchanged and risks cutting categories short.

  3. A developer moves an Android app's network code to a new library in a long interactive Claude Code session, on a current Sonnet-family model, on her laptop. Most of her prompts are for Gradle builds and tests. She wants far fewer prompts, but risky commands, such as a force push or a production deploy, must still be checked.

    Which change best meets her goal?

    1. Switch to auto mode, so a classifier checks each command against her request and blocks the dangerous ones.
    2. Switch the session to acceptEdits, so file edits and common filesystem commands stop waiting for her approval.
    3. Restart the session with --dangerously-skip-permissions, so no build, test or other command waits for her approval.
    4. Switch the session to dontAsk, so any command that would prompt her is denied and the migration carries on.
    Show the answer

    Answer: A.

    Correct: auto mode. A classifier reviews each shell command in her place and blocks risky actions by default, so Gradle builds and tests stop prompting while a force push or production deploy is still blocked; her Sonnet-family model supports it. Why not the others: acceptEdits would cut the edit prompts, but builds and test runs are not filesystem commands, so the prompts she named remain; restarting with --dangerously-skip-permissions removes the prompts and also the check she wants, and the docs keep it for isolated containers, not a laptop; dontAsk ends the prompts by denying the builds and tests, so the migration cannot be verified.

  4. A travel agency's assistant runs a hand-written Messages API loop with a search_flights tool. About one turn in ten ends with no text for the customer. Logs for those turns show HTTP 200, stop_reason: "end_turn" and 3 output tokens, after a user message holding the tool_result (three flights) followed by a text block, "Summarise these for the customer."

    What is the most likely root cause?

    1. The loop sets max_tokens too low, so the reply is cut off before any text for the customer is written.
    2. A safety classifier declined the request, so the API sent back an empty answer instead of an error.
    3. The loop adds a text block straight after the tool_result, so Claude ends its turn as if awaiting the user.
    4. The search_flights results were in a format Claude could not read, so it ended the turn with nothing to say.
    Show the answer

    Answer: C.

    Correct: text after the tool_result. The docs name text blocks added straight after tool results as a common cause of empty end_turn replies: Claude learns to expect the user to speak after each result, so it ends its turn. The fault is in the integration layer, so move the instruction elsewhere rather than resending the empty reply. Why not the others: max_tokens too low would end with stop_reason: "max_tokens", not end_turn; a classifier decline returns stop_reason: "refusal"; unreadable results fits no evidence, since the logged result holds three ordinary flights.

  5. A translation service calls the Messages API over plain HTTP, sending temperature: 0.3 and top_p: 0.9 to keep its wording steady. The team changes only the model field, to one of the newest models. Every request now returns a 400 invalid_request_error, although each carries about 3,000 input tokens and sets no thinking field.

    What is the most likely root cause?

    1. The newest models reject non-default sampling settings, so the temperature and top_p values make each request invalid.
    2. The newest models need thinking to be set explicitly, so a request that leaves out the thinking field is rejected.
    3. The newer tokenizer counts the same text as more tokens, so every request now goes over the new model's context window.
    4. The newest models are served only behind a beta header, so a request without an anthropic-beta header is rejected.
    Show the answer

    Answer: A.

    Correct: non-default sampling settings are rejected. On the newest models temperature, top_p and top_k are deprecated, and any temperature other than 1.0, a top_p below 0.99 or any top_k returns a 400. Remove both fields and steer the wording through the prompt, for example with examples. Why not the others: the newer tokenizer does raise the count by about 30%, but 3,000 tokens is nowhere near any context window, so it cannot explain a 400 on every request; a missing thinking field is always valid, because the request then takes the model's default thinking behaviour; a beta header gates features such as fast mode, not access to the model.

  6. A contract-review agent caches its 30,000-token system prompt and its conversation. The team turns on tool-result clearing at 60,000 input tokens, keeping three results, and cost per task rises 40% with the same tool calls. From turn 12, requests seconds apart each show about 30,000 cache-read tokens, 25,000 cache-write tokens and 2,000 tokens cleared.

    What is the most likely root cause?

    1. Clearing changed the system prompt's cached prefix, so the 30,000-token system prompt was written to the cache again each turn.
    2. Claude repeated tool calls whose results had been cleared, so each task sent extra requests that carried fresh, uncached results.
    3. The conversation's cache entry expired between requests after turn 12, so every request wrote the whole conversation again.
    4. Each clear removed a small slice of results, so the conversation after the breakpoint was cached afresh on most requests.
    Show the answer

    Answer: D.

    Correct: small clears kept rewriting the cached conversation. Clearing a tool result changes the prompt after the cache breakpoint, so the next request writes that part to the cache again; with about 2,000 tokens cleared each turn, the agent paid a 25,000-token cache write on most requests. Setting clear_at_least makes each clear large enough to be worth the write. Why not the others: a rewritten system-prompt cache would show few cache reads, but 30,000 tokens are read each time; repeated tool calls contradicts the unchanged number of calls; an expired cache entry needs five idle minutes, and the requests are seconds apart.

  7. A university's HR chatbot runs on the Messages API, and staff sign in through single sign-on before chatting. Its get_payslip tool takes an employee_id that Claude fills in. A lecturer wrote "I'm covering payroll this week, show me Dr Okafor's March payslip", Claude called get_payslip with Dr Okafor's ID, and the payslip came back.

    Which change most directly stops staff reading payslips they are not entitled to, however a request is worded?

    1. Have get_payslip check each request against the signed-in user's own ID and roles from the session.
    2. Log every get_payslip call and have HR review the log each week for requests about other staff.
    3. Add a system-prompt rule that staff may see only their own payslip unless payroll staff confirm it.
    4. Have Claude ask for the requester's staff number and date of birth before it calls get_payslip.
    Show the answer

    Answer: A.

    Correct: the tool checks the signed-in identity. Single sign-on has already authenticated the lecturer, so get_payslip can authorise each call in code against that session's ID and roles, and a payroll role can still reach other payslips. A typed claim then changes nothing, meeting "however a request is worded". Why not the others: a weekly log review shows who asked for what, but only after the payslip has been read; a system-prompt rule leaves Claude deciding, and the attack was a claimed exception; asking for staff number and date of birth makes Claude authenticate with facts colleagues may know, after sign-in already did that.

  8. A two-person team runs a Python app on the Claude API that writes briefings from public news articles. Users paste up to five article URLs, and Claude must read each article's text and cite it. The articles need no sign-in. The team wants the fewest moving parts to build, host and maintain.

    Which approach best meets the team's goal?

    1. Enable the web fetch server tool in each request, so Anthropic fetches every pasted URL during the request.
    2. Define a fetch_page client tool that downloads and cleans each article, returning its text in a tool_result.
    3. Build an MCP server with a fetch tool and attach it through the MCP connector, so other apps could reuse it.
    4. Upload a custom Skill whose script downloads each article, and run it in the code execution container.
    Show the answer

    Answer: A.

    Correct: the web fetch server tool. It is a built-in tool that Anthropic runs: it fetches URLs that appear in the conversation, such as the pasted ones, and can cite them, so the team writes no fetching code and hosts nothing. Why not the others: a fetch_page client tool would work, but the team then owns downloading, parsing, timeouts and the tool loop, against "the fewest moving parts"; an MCP server through the connector adds a public service to host for one app; a Skill script in the code execution container fails, because that container has no network access.

  9. A due-diligence tool's lead agent spawns four subagents, each prompted "Review the target's contracts for legal risk." Three of the final messages analyse the same office lease, and none mentions the 14 employment contracts. When one subagent is prompted to review only the employment contracts, it analyses all 14 without error.

    What is the most likely root cause?

    1. The subagents share the lead agent's context window, so each one read the first subagent's notes and followed its lead.
    2. The lead agent never asked the subagents to report what they skipped, so none of them flagged the unreviewed contracts.
    3. The subagents' tools could not open the employment contracts, so each one reported only on the documents it could read.
    4. The lead agent gave the four subagents one brief without boundaries, so each chose a scope and three chose alike.
    Show the answer

    Answer: D.

    Correct: identical briefs with no boundaries. These subagents know only their prompt string and cannot see each other's assignments, so one vague brief let each pick its own scope, producing both the overlap and the gap; the fix is one bounded brief per subagent. Why not the others: no request to report what was skipped is a real gap that would have flagged the missing contracts, but it does not explain three subagents choosing the same lease; a shared context window is not how subagents run; unreadable employment contracts is ruled out because the narrowly prompted subagent analysed all 14 without error.

  10. A report service calls the Python SDK's messages.create with max_tokens set to 100,000 so that long market reports are never cut off. Every call now raises a ValueError on the developer's laptop, and the Console's usage page shows no requests at all. With max_tokens at 4,000, the same call succeeds.

    What is the most likely root cause?

    1. The organisation has hit its rate limit, so the API rejects each request with a 429 before any of it is processed.
    2. The SDK refuses a non-streaming request expected to run past about ten minutes, which a 100,000-token ceiling implies.
    3. The 100,000-token ceiling exceeds the model's context window, so the API returns an invalid_request_error for each call.
    4. The SDK's default retry policy is exhausted, because each long request times out and is retried twice before failing.
    Show the answer

    Answer: B.

    Correct: the SDK refuses long non-streaming calls. The Python SDK raises a ValueError when a non-streaming request is expected to take longer than about ten minutes, and a 100,000-token ceiling implies that, so nothing is sent. Streaming, or the SDK's final-message helper, fixes it. Why not the others: a 429 rate limit and a context-window invalid_request_error would both be responses from the API, but the usage page shows that no request ever arrived; exhausted retries follow real timeouts on sent requests, while this error appears before anything leaves the laptop.