sample questions

10 of the 90 questions, spread across the domains. Open one to see the answer and why.

  1. Analysts now send the research system many quick questions, such as which company acquired a named startup and when. Every query runs web search, document analysis, synthesis and report generation, taking about three minutes. Quick questions must be answered in under 20 seconds, and broad topics must still get the full cited report.

    Which change best meets the 20-second target while preserving full reports for broad topics?

    1. Cache every finished report and return the cached report whenever a new query closely matches an earlier one.
    2. Have the coordinator assess each query, invoking only web search for one-fact questions and the full pipeline for broad topics.
    3. Remove report generation from the pipeline and have synthesis return a short direct answer for every query.
    4. Cap every subagent at three tool calls, so that each of the four pipeline stages finishes well inside the 20-second target.
    Show the answer

    Answer: B.

    Correct: the coordinator selects subagents per query. A one-fact question needs one web search subagent and no document analysis, synthesis or report, while broad topics keep the full pipeline, which meets both halves of the stem. Why not the others: removing report generation would make quick questions fast, but a short direct answer for every query removes the full report that the stem says broad topics must keep; a three-call cap starves broad research of the searches it needs and still runs four stages; caching reports helps only repeated questions, and most quick questions are new.

  2. An engineer asks how a declined card payment becomes a dunning email in the legacy billing system. The only clue is the log line Payment declined, scheduling retry. The agent must finish a refactor in the same session afterwards, so its context window cannot fill with files unrelated to this flow.

    What is the most effective first step?

    1. Grep for the log line's text, Read the file that raises it, and follow that file's imports to the email code.
    2. Read every file under src/payments/ and src/email/ before running any search, then trace the whole flow from what those two directories contain.
    3. Glob for **/*dunning* and Read each matching file, then piece the flow together from what those files contain.
    4. Run the payment test suite through Bash and read the output of every test that mentions a declined card.
    Show the answer

    Answer: A.

    Correct: Grep the log line, then follow imports. An error message is file content, so Grep finds the code that raises it, and reading only the files its imports lead to builds the flow incrementally while keeping the context window free for the refactor. Why not the others: reading both directories would probably cover the flow, but it loads dozens of unrelated files, which the stem rules out; globbing for dunning matches file names, so it misses the payment side and still reads every match; the payment test suite shows behaviour, not which code path turns a decline into an email.

  3. A pipeline step asks Claude to fix a failing lint check, then runs claude -p --continue "Review the fix you just made" before posting. The review approves 97% of fixes, yet humans later find bugs in one fix in five. The approvals often repeat the fix's own reasoning, such as "the null check was redundant, as planned".

    Which change most directly addresses the cause of the missed bugs?

    1. Run the review as a separate claude -p call that receives only the diff and the CLAUDE.md review criteria.
    2. Keep the continued session, but add a detailed checklist to the prompt that tells Claude to challenge each of its decisions.
    3. Add a confidence field to the --json-schema output and send approvals scored below 0.8 to a human reviewer.
    4. Raise --max-turns for the review step so the continued session can re-read every file that the fix touched.
    Show the answer

    Answer: A.

    Correct: a separate review instance. The root cause is missing session context isolation: the continued session still holds the reasoning that produced the fix, and the approvals quote it back. A new claude -p call sees only the diff and the criteria, so it judges the code as written. Why not the others: a stricter checklist would catch some bugs, but the stem shows the reviewer reusing the fix's own reasoning, and the same session keeps that reasoning in context; a confidence threshold asks the same anchored session to rate itself; more turns lets it re-read files, not question its own assumptions.

  4. Contracts arrive around the clock and must reach the ledger within 36 hours of arriving. Extraction runs through the Message Batches API, and after each batch ends the team needs 3 hours to validate the results and route low-confidence records to human review.

    Which submission schedule meets the SLA with the fewest batches per day?

    1. Submit a batch every 12 hours, because 12 hours of waiting plus 24 hours of processing is exactly 36.
    2. Submit a batch every 8 hours, so the worst case is 8 hours waiting, 24 processing and 3 reviewing.
    3. Submit a batch every 4 hours, so the worst case is 4 hours waiting, 24 processing and 3 reviewing.
    4. Submit one batch a day at midnight, because most batches finish in well under an hour of processing.
    Show the answer

    Answer: B.

    Correct: every 8 hours. The worst case is the submission interval plus the full 24-hour window plus the 3 review hours: 8 + 24 + 3 = 35, inside 36, with three batches a day. Why not the others: every 4 hours also meets the SLA (31 hours), but it needs six batches a day, and the stem asks for the fewest; every 12 hours reaches 36 before the 3 review hours, so 39; one batch a day treats the typical hour as a bound, when the worst case is 24 + 24 + 3 = 51 hours.

  5. A customer raises four issues. Each is investigated by a subagent that returns about 2,500 tokens of narrative reasoning ending in a conclusion. The agent that writes the reply has about 6,000 tokens for findings, so the fourth result is cut off. Every statement in the reply must name the order record it rests on.

    Which change best fits all four findings into the budget while preserving the record behind each statement?

    1. Add a summarisation step that condenses each subagent's narrative to about 500 tokens before the reply agent reads it.
    2. Put a key-findings summary at the start of the combined input, and give each issue its own section header.
    3. Change each subagent to return structured findings, such as the key fact, the record cited and a relevance score.
    4. Have each subagent trim the lookup_order results it receives to the return-relevant fields before it reasons.
    Show the answer

    Answer: C.

    Correct: structured findings from each subagent. The budget overflows because the upstream agents send verbose content and reasoning chains to a downstream agent with little room. Key facts, citations and a relevance score fit in under a hundred tokens per issue and keep the record each statement needs. Why not the others: a summarisation step would also fit the budget, but a paraphrased narrative can drop the order record the stem requires, and it adds a call; key findings first with headers changes where findings sit, not what they cost, so the fourth is still cut off; trimming lookup_order results shrinks what the subagents read, not the narrative they return. "Must name the order record" decided it.

  6. For the topic "remote work's effect on city economies", the coordinator sends three web search subagents the same brief: "Research remote work and cities." Two return near-identical findings on office vacancy, and nobody covers tax revenue or transit ridership. Every subagent succeeded. Every exchange between agents must appear in the coordinator's log.

    Which TWO changes to the coordinator best fix both the duplication and the missing coverage?

    1. Add a deduplication step before synthesis that drops any finding repeating another subagent's finding.
    2. Give each search subagent a distinct subtopic, such as vacancy, tax revenue or transit, with explicit boundaries.
    3. Let the three search subagents share a scratchpad file, so each one can see what the others have covered.
    4. After synthesis, check the draft against the topic's parts and re-delegate targeted queries for any missing part.
    5. Raise the number of search subagents from three to six, each given the same brief, to widen the coverage.
    Show the answer

    Answer: B and D.

    Correct: distinct subtopics, and a gap check with targeted re-delegation. The root cause is an overly narrow, undivided decomposition: subagents cannot see each other's briefs, so only the coordinator can partition the scope, and only a gap check after synthesis catches a part nobody was given. Why not the others: a shared scratchpad could reduce overlap, but subagents would exchange data outside the coordinator's log, which the stem forbids; deduplication removes the repeat after it was paid for and adds no coverage; six identical briefs multiply the duplication.

  7. A tax change needs two edits. In eu-vat.ts, all four lines reading vatRate: 0.20, must become 0.21. In uk-vat.ts, generated code repeats one block three times, and only the first block's rate must change. The blocks are identical line for line, so adding neighbouring lines to old_string still matches all three.

    Which TWO moves best make both changes?

    1. Run sed through Bash on both files, replacing every vatRate: 0.20 with vatRate: 0.21 in a single command.
    2. Call Edit on eu-vat.ts with replace_all set to true, because every one of the four lines should change.
    3. Widen old_string in uk-vat.ts with the lines above the rate until the Edit matches only one block.
    4. Call Edit on eu-vat.ts four times with the same old_string, changing one of the four lines per call.
    5. Read uk-vat.ts in full, then Write the whole file back with only the first block's rate changed.
    Show the answer

    Answer: B and E.

    Correct: replace_all for eu-vat.ts, and Read + Write for uk-vat.ts. Edit needs old_string to match exactly once. When every occurrence should change, replace_all: true is the intended exception, and when nothing distinguishes the one block, a full Read then Write is the reliable fallback. Why not the others: widening old_string is the usual first fix, but the stem says neighbouring lines still match all three blocks; calling Edit four times fails because each call's old_string is non-unique; sed on both files is right for eu-vat.ts but also changes all three uk-vat.ts blocks.

  8. The review job fails the pull-request check only when claude -p exits non-zero. Last week a pull request merged with a finding marked "severity": "blocking" in structured_output. Twice this month the posting script broke because Claude reworded its prose summary. Runs that fail on authentication must still fail the check.

    Which change best blocks merges on blocking findings while keeping failed runs visible?

    1. Tell Claude to end its prose summary with the word BLOCKED, and have the workflow grep the .result field for it.
    2. Keep the exit-code check, and add a jq step that fails the job when structured_output has any blocking finding.
    3. Switch to --output-format stream-json so that the workflow can cancel the job at the first blocking event.
    4. Add --bare so that every run loads the same configuration and reports its blocking findings more consistently.
    Show the answer

    Answer: B.

    Correct: a findings gate beside the exit code. Exit code 0 means the run finished, not that the code is clean, so the gate has to read the parsed findings in structured_output. Keeping the exit-code check leaves failed runs, such as authentication errors, visible. Why not the others: a grep of .result for BLOCKED would work on most runs, but it depends on prose, and the stem says rewording already broke the script twice; stream-json changes how output arrives, not what decides the gate; --bare changes what loads at startup, not the merge decision.

  9. A nightly batch of 8,000 support emails hit the 24-hour limit: 7,400 succeeded and 600 expired. The 600 emails arrived about 5 hours before the batch was submitted, and the SLA is 30 hours from arrival, so about one hour remains.

    What is the most effective way to handle the 600 expired requests?

    1. Resubmit the 600 in a new batch by custom_id, because most batches finish in under an hour.
    2. Resubmit all 8,000 emails in a new batch, so that every result comes from the same run.
    3. Send the 600 through the synchronous Messages API now, matching each result back by custom_id.
    4. Chunk the 600 emails into smaller pieces and resubmit them, because expired requests were too large.
    Show the answer

    Answer: C.

    Correct: the synchronous API for the 600. Only about one hour of the SLA remains, and the synchronous API answers in seconds, at full price for just 600 requests. Why not the others: a new batch of the 600 is the normal move and would usually finish in time, but a batch guarantees nothing shorter than 24 hours, and the stem leaves one hour; all 8,000 pays again for 7,400 results already in hand; chunking fixes oversized documents, but these requests expired because processing ran out of time, not because they were too large.

  10. To save tokens, your client sends Claude only the system prompt and the last six turns. In a 22-turn session about two orders, the customer says "Yes, apply that to the second one", and the agent asks which offer and which order they mean. The 15% discount offered in turn 5 appears nowhere in the request.

    Which TWO changes most directly restore coherence without letting the request grow without limit?

    1. Send only the turns that mention an order number, so small talk is dropped and order details stay.
    2. Move to a model with a larger context window, so the client can send the last twelve turns.
    3. Send the complete conversation history on every request, and summarise older turns when it nears the budget.
    4. Instruct the agent to ask the customer to restate the order and offer whenever a reference is unclear.
    5. Keep a per-order facts block with order ID, offer and status that your code sends outside the history.
    Show the answer

    Answer: C and E.

    Correct: complete history plus a per-order facts block. The API is stateless, so a turn left out of the request never happened for the model. Sending every turn restores what "that" refers to, and summarising older turns keeps the size bounded. A summary would blur the 15%, so a facts block your code rebuilds keeps the offer and its order exact. Why not the others: asking the customer to restate would recover the details, but it hands the agent's memory failure to the customer on every unclear reference; only turns with an order number still drops turn 5, where the offer was made in words; a larger window moves the cut-off without removing it. "Most directly" and "without limit" decided it.