Task statement 5.2 · Model selection and optimization

5.2 · Technical Fundamentals

In Shelfwise, the course example, no engineer writes an HTTP request to Claude by hand: the Python Agent SDK sends each Messages API call, retries it when it fails and reads the streamed reply, while the service streams the answer on to the staff widget. Three things decide how the service behaves when the API is busy: what the SDK does, what it leaves to you, and which transport each link uses.

Why it's tested

Two reasonable instincts cost time here. One treats the Claude API as just another REST endpoint and calls it with a plain HTTP library, which works at first but leaves the team rebuilding retries, error handling and stream parsing that the SDKs already ship. The other looks for a WebSocket to stream Claude's answer, but the API streams over server-sent events on an ordinary HTTP response, so a WebSocket is a choice only on links your own application owns.

Start hereTwo terms this section adds

client SDK
An official library, such as anthropic for Python or @anthropic-ai/sdk for TypeScript, that wraps the Claude REST API in one language's own types and method calls.
WebSocket
A protocol that starts as an HTTP request, switches that connection over, and then lets either side send messages at any time until one of them closes it.

KnowledgeWhat a client SDK adds over the REST API

A client SDK sends the same HTTP requests you could build yourself, and what it adds is the code around each request. With no key passed, the client reads ANTHROPIC_API_KEY from the environment, and it sets the anthropic-version header for you. Requests and responses are typed, as TypedDicts and Pydantic models in Python and as TypeScript definitions, so an editor can flag a misspelt field before the API returns a 400. A failed call raises an error class named for its cause, such as RateLimitError for a 429 or APIConnectionError when the server could not be reached, so your code catches the cases it can handle and lets the rest propagate. Every response object also carries _request_id, copied from the request-id header that 2.2 logs for support.

The client also manages failure and time. It retries the transient failures that 2.4 lists, twice by default, and a timed-out attempt counts as one of them. In the Python SDK each attempt may run for 10 minutes by default, and you change that with timeout on the client or for one call with with_options(), alongside max_retries (maxRetries in TypeScript). At worst, when every attempt times out, your code waits the per-attempt timeout times the number of attempts, plus the backoff between them.

For streaming, messages.create(stream=True) returns the raw events and keeps nothing, while messages.stream() gives you text_stream for the text as it arrives and get_final_message() (finalMessage() in TypeScript) for the accumulated Message, the same object a non-streaming call returns. List endpoints are paginated, and a plain loop over the result fetches further pages as needed. The beta tool runner (1.2) runs the tool loop for functions you mark with @beta_tool. The Agent SDK works a level higher, running the whole agent loop with built-in tools, and 2.5 covers what each one loads.

KnowledgeDropping to raw HTTP

Official client SDKs exist for Python, TypeScript, C#, Go, Java, PHP and Ruby, so raw HTTP is mainly for a runtime none of them fits. Choosing it means writing what the SDK did. You send x-api-key, anthropic-version and content-type yourself, decide which statuses to retry and how long to wait, and read the error body's type and request_id. A streamed response has to be read event by event. Your reader skips ping events, tolerates event types added later and joins the deltas into a message. It also treats an error event that arrives after the 200 as a failure, because an overloaded API can report itself mid-stream, as 2.3 notes. For long requests the docs also advise setting TCP keep-alive, which the SDKs already do, so a network that drops idle connections does not cut the response off.

KnowledgeTransports: server-sent events and WebSockets

Server-sent events (SSE) are the WHATWG HTML standard's way for a server to push data over HTTP. The client sends one request. The server answers with the text/event-stream media type and keeps writing named events down that response. Data flows only from server to client, so the client says more only with a new request. A WebSocket, defined in RFC 6455, opens with an HTTP request carrying Upgrade: websocket. The server answers 101 Switching Protocols, and from then on the same TCP connection carries frames in both directions until either side closes it. What decides between them is whether one side must send while the other is still sending.

The Messages API streams with SSE, and so does the Managed Agents session event stream. WebSockets appear on links your own application owns. The Agent SDK's hosting guide says a container serving long-running sessions exposes an HTTP or WebSocket port to its own clients. Claude Code can also reach an MCP server over a WebSocket with "type": "ws", although WebSocket is not one of the MCP specification's standard transports, which 8.2 covers. Shelfwise's widget sends one question and then only receives, so the service answers its POST with an SSE stream of answer text.

Interactive · how the widget's link changes while the link to Claude stays the same

A sequence diagram read from top to bottom, with three columns: the staff widget in a browser, the Shelfwise service running the Agent SDK, and the Claude API. In the default state the widget posts its question, the service answers 200 with text/event-stream, calls the Claude API with POST /v1/messages and streaming on, receives 200 text/event-stream and content_block_delta events, forwards the text to the widget as data lines, and ends both streams after message_stop. The WebSocket state replaces only the widget's link: an Upgrade request, a 101 response, and frames in both directions. Staff widget BROWSER Shelfwise service PYTHON AGENT SDK Claude API MESSAGES API POST /v1/messages · stream 200 · text/event-stream event: content_block_delta ×n event: message_stop POST /ask · {question} 200 · text/event-stream data: {answer text} ×n response ends one way: a new question is a new POST GET /chat · Upgrade: websocket 101 Switching Protocols frame: {question} frame: {answer text} ×n frame: at any time connection stays open two way: either side sends until one closes Ember arrows: a link Shelfwise owns. Blue arrows: the link to Claude.

Choose how the widget's link to the service is built. The service's link to the Claude API doesn't change.

Server-sent events, Shelfwise's choice: the widget POSTs its question and reads the answer from that response as a text/event-stream. Data flows only from the service, so a new question is a new request.

WebSocket: the widget's first request upgrades the connection with a 101 response, and either side can then send frames at any time. Shelfwise doesn't need this, because the widget has nothing to send while an answer streams.

Time runs downward. Only the ember link to the widget changes between the two states; the blue link to the Claude API is one POST answered by server-sent events in both.

More precise than the guide. The guide lists websockets among the engineering practices for integrating with Claude. The Messages API streams over server-sent events and the Claude API offers no WebSocket endpoint (as of 2026-10-04), so WebSockets appear only on links your application owns. Exam answer: Claude's responses reach your code as server-sent events on an HTTP response, and a WebSocket suits only a link where both ends must send at any time.

KnowledgeConnections, keys and timeouts in a long-lived service

A service makes many calls, so create one client when the process starts and share it. The Python SDK sends requests through an httpx2 client, which keeps a pool of connections and reuses them for the same host. Later requests therefore skip the TCP and TLS handshakes, while a client built inside each request handler pays for them on every call. Keep the key in the environment, where both the client SDKs and the Agent SDK read ANTHROPIC_API_KEY. Keep it out of source code and browser pages: the TypeScript SDK refuses to run in a browser unless you set dangerouslyAllowBrowser, because anyone using the page could read the key. 7.4 covers storing and rotating keys. Set each timeout to what the caller will wait for, and stream any request that may run long, because the SDKs refuse a non-streaming request expected to take more than about ten minutes, as 2.3 explains.

SkillTuning the Agent SDK's retries for an answer at the counter

The Agent SDK has its own retry layer, because its API calls are made by a Claude Code subprocess. That subprocess retries transient failures up to 10 times with exponential backoff (CLAUDE_CODE_MAX_RETRIES), gives each attempt 10 minutes (API_TIMEOUT_MS), and reports each retry as an api_retry system message. Shelfwise met those defaults one Saturday:

# Saturday 11:20: the API answers 529 overloaded for a few minutes
"M8 hex bolts, Leeds?" → widget spins for about 2 minutes; customer leaves
log: api_retry attempt 1 … 7 of 10, error "overloaded", error_status 529
# cause: defaults CLAUDE_CODE_MAX_RETRIES=10, API_TIMEOUT_MS=600000
# fix: ClaudeAgentOptions(env={"CLAUDE_CODE_MAX_RETRIES": "2", "API_TIMEOUT_MS": "20000"})
next overload → error after 2 retries; widget: "Stock check busy, check the shelf"

The defaults suit an unattended job that should outlast an outage, and their worst case, the timeout times eleven attempts, is nearly two hours. A customer at the counter waits a minute or two at most. The service sets two retries and 20 seconds per attempt. No Shelfwise request comes near 20 seconds. The worst case is then about a minute plus backoff, while a brief 529 is still absorbed. The service also forwards each api_retry to the widget as "Busy, retrying", so staff know whether to wait. Raising the client SDK's max_retries would change nothing here, because the Agent SDK's calls go through its Claude Code subprocess, which takes these settings from ClaudeAgentOptions.env, not from a client SDK.

Before writing code around a Claude call, ask: does the SDK already do this? Yes (auth, retries, timeouts, typed errors, stream parsing) → configure the SDK, sizing its retries and timeout to whoever waits on the answer. No (the link to your own clients) → choose the transport: SSE when only the server sends after the request, a WebSocket when both sides must send at any time.
Takeaway
  • Claude's API streams as server-sent events over one HTTP response; WebSockets belong on links your own application owns.
  • A timed-out attempt is retried like any transient failure, so at worst, when every attempt times out, the wait is the timeout times the attempts, plus backoff.
  • Create one client per process and let it read ANTHROPIC_API_KEY; an Agent SDK service sets retries and timeouts through CLAUDE_CODE_MAX_RETRIES and API_TIMEOUT_MS.

check yourself

0 / 4 answered

A Python service writes product descriptions with the Messages API, without streaming, and each one takes about 35 seconds to generate. A developer set timeout=20.0 on the service's shared Anthropic() client and left every other option at its default. Every call now raises APITimeoutError, and each error reaches the log about a minute after its call began.What is the most likely root cause?

Select one.

A coding school is adding a tutor to its browser-based editor. While Claude's feedback on a submission streams in, the page keeps sending the student's edits, several times a second, and the server must be able to push a teacher's note at any moment. The school wants no new HTTP request per edit, and its API key must never reach the browser.Which design best meets these requirements?

Select one.

A FastAPI service with plain synchronous def handlers builds a new Anthropic(api_key="sk-ant-…") client inside each request handler, with the key pasted into the source. At 20 calls a second, profiling shows each call spending about 150 ms opening a new TLS connection to the API, and a security review found the key in git history. The key has since been rotated.Which TWO changes most directly remove the per-call connection cost and keep the new key out of the code?

Select two.

A team's Rust service calls the Messages API over raw HTTP with streaming on, because no official SDK covers Rust. Its reader acts only on content_block_delta and message_stop events and silently skips every other type. On a busy afternoon some answers stop mid-sentence: each had status 200, partial text and no message_stop, and the HTTP client logged a normal close.What is the most likely root cause?

Select one.

sources

  1. Claude Platform Docs, "SDKs, CLI, and libraries": client SDKs for Python, TypeScript, C#, Go, Java, PHP and Ruby with type safety and built-in streaming, retries and error handling; the Agent SDK provides the agent loop. Checked 2026-10-04.
  2. Claude Platform Docs, "Python SDK": ANTHROPIC_API_KEY as the default key; TypedDict requests and Pydantic responses; the error classes by status; _request_id; retries and max_retries; the 10-minute default timeout, with_options(), timeouts retried twice and APITimeoutError; the long-request ValueError and TCP keep-alive; messages.stream(), get_final_message() and create(stream=True); auto-pagination; @beta_tool and the tool runner; requests sent with httpx2. Checked 2026-10-04.
  3. Claude Platform Docs, "TypeScript SDK": type definitions for every parameter; maxRetries and timeout; finalMessage(); browsers disabled by default unless dangerouslyAllowBrowser is set, because it exposes the key. Checked 2026-10-04.
  4. Claude Platform Docs, "Streaming messages": "stream": true over server-sent events; named events; ping events; error events in the stream; handle unknown event types gracefully; the SDK's accumulation helpers. And "Claude API errors": the error body's type and request_id; the request-id header; TCP keep-alive for direct integrations. Checked 2026-10-04.
  5. Claude Platform Docs, "Session event stream": the stream is requested with accept: text/event-stream. Checked 2026-10-04.
  6. Claude Code Docs, "Hosting the Agent SDK": a long-running container exposes an HTTP or WebSocket endpoint; the subprocess reads ANTHROPIC_API_KEY; outbound HTTPS to the API. And "Connect Claude Code to tools via MCP": remote WebSocket servers with "type": "ws", a persistent two-way connection. Checked 2026-10-04.
  7. Claude Code Docs, "Agent SDK reference - Python", handle slow or stalled API responses: API_TIMEOUT_MS (default 600000) and CLAUDE_CODE_MAX_RETRIES (default 10) passed through ClaudeAgentOptions.env; worst case the timeout times the retries plus one. And "Error reference", automatic retries: overloaded responses retried with exponential backoff; and "Run Claude Code programmatically": the system/api_retry event and its fields. Checked 2026-10-04.
  8. Model Context Protocol, "Transports" (2026-07-28): the standard transports are stdio and Streamable HTTP; custom transports are allowed. Checked 2026-10-04.
  9. WHATWG, HTML Living Standard, "Server-sent events": servers push data to web pages over HTTP with the text/event-stream type. And IETF, RFC 6455, "The WebSocket Protocol": two-way communication; an opening handshake with Upgrade: websocket and 101 Switching Protocols, then message framing over TCP. Checked 2026-10-04.
  10. HTTPX2 documentation, "Clients": a client instance uses connection pooling and reuses the underlying TCP connection for requests to the same host, with no repeated handshaking. Checked 2026-10-04.
  11. Shelfwise's widget link, the Saturday overload, the hex-bolt question, the retry log and the 20-second setting are this course's example details.