Developer Foundations · 2.6% of the exam

eval, testing and debugging

This is the smallest domain, about one scored question in forty. The guide's title names evals and testing, but its one objective is debugging and error handling: telling one kind of failure from another, choosing the recovery each needs, and finding which layer to fix.

Course example

This domain keeps following Shelfwise, the internal stock-question tool: a long-lived Python service built on the Agent SDK, the Anthropic library that runs Claude in a tool loop, whose agent answers shop staff from one stock feed through stock_mcp, its one MCP server, a program that offers tools to Claude applications over MCP, the Model Context Protocol.

In Domain 4 Shelfwise has its one debugging incident. From 10:00 one Monday, every Wakefield answer read "No record" for products on the shelf, while Leeds answers carried counts. The team read one answer's steps in order and found the first wrong value in the tool's result: the feed had renamed the branch "Wakefield Westgate", and stock_mcp matched branch names exactly. The fix maps the names and reports an unknown branch as a tool error (4.1).

four checkpoints, read in orderfirst wrong value: the tool's resultfix in stock_mcp, not the promptunknown branch: a tool error
0 errorsraised while every Wakefield answer was wrong. The run reported success, and the answer check passed each "No record" line, because an empty result looked like a product missing from the feed. A retry would have repeated it. Since the fix, a question about an unknown branch gets "Stock check failed, check the shelf".

Start hereThe words used throughout this domain

stop_reason
The field in a successful Claude API response that says why Claude stopped: it finished its turn, wants a tool run, hit your output ceiling, or something else (2.3, 6.3).
tool_use, tool_result
The block in which Claude asks for a tool call, and the block your code sends back with the outcome (2.3).
is_error, isError
The flag that marks a failed tool call's result: is_error on a Claude API tool_result block, isError on an MCP tool result (8.1, 8.2).
hook, PostToolUse
Your own code that runs at a fixed event in Claude's agent loop. A PostToolUse hook runs after a tool call succeeds. Shelfwise's lookup log is one, an Agent SDK callback (1.2). Since the Wakefield fix (4.1), it also runs under PostToolUseFailure, after failed calls.
529, 429
HTTP statuses for transient failures: a 529 overloaded_error when the API is busy, and a rate-limit 429 when you send faster than your limits allow. Both are worth retrying after a wait (2.4, 5.2).
trace
The record of one run's steps in order: each request and its status, each tool call's input and result, and the final answer (4.1).

task statements

flashcards

0 / 3 known
Card 1 of 3