Task statement 4.5 · Prompt engineering and structured output

Design efficient batch processing strategies

The extraction system has two workloads. Suppliers upload an invoice and wait on the page while it is checked. Separately, about ten thousand scanned invoices and contracts a day must be in the ledger within 30 hours of arriving. Your manager proposes moving both to the Message Batches API to halve the cost.

Why it's tested

The 50% saving makes batch look like a free win, so the tempting answer moves every workload onto it. That fails wherever someone is waiting, because a batch promises nothing shorter than 24 hours, and a supplier on the upload page cannot wait that long. The second tempting move is to resend the whole batch when a few requests fail, which pays again for every request that already succeeded and repeats the failures nobody fixed.

Start hereThree terms this section adds

custom_id
Your identifier on each request in a batch, such as inv-20260921-00417. Results come back in any order, so it is how you match each result to its document.
latency-tolerant
A non-blocking, latency-tolerant workload is one where nothing stops while it runs, so a result tomorrow morning is as good as one in a minute.
blocking workflow
A person or a pipeline cannot continue until the result arrives, such as a supplier on an upload page or a pre-merge check.

KnowledgeHow a batch runs

The Message Batches API takes many Messages requests in one call, each with a custom_id and the same params a normal Messages call takes. The requests are processed asynchronously, and all usage is charged at 50% of standard prices. Most batches finish in under an hour, but results become available only when every request has finished or 24 hours have passed, and requests still unprocessed at 24 hours expire. "Usually within an hour" is therefore not a design number, because the API guarantees no latency shorter than the 24-hour processing window.

  1. You create the batch, and its processing_status is in_progress.
  2. You poll until the status becomes ended.
  3. You download the results, which stay available for 29 days.
  4. You match each result by custom_id, because the order is not guaranteed.
  5. You resubmit only the requests that did not succeed.

Choosing the API is a question about who waits. The supplier's upload check is a blocking workflow: the supplier stays on the page, so the answer must arrive in seconds, which only the synchronous Messages API does. The nightly ledger load is non-blocking and latency-tolerant, so a batch that takes eight hours costs nothing and saves half the price. Polling every minute does not make a batch finish sooner, and a fallback to the synchronous API after a timeout means you build both paths and still end up waiting on the synchronous one.

Blocking

Synchronous Messages API

The supplier's upload check.

Result: full price, answer in seconds.

Non-blocking, latency-tolerant

Message Batches API

Ten thousand scanned documents a day, 30-hour SLA.

Result: half price, results within the SLA.

Scenario switchThe guide's own examples come from CI. A pre-merge check on pull request #812 holds the merge until it reports, so it is blocking and stays on the synchronous API: sent in a batch at 10:05, its result could arrive at 10:20 or at 10:04 the next day. Nightly test generation for the day's changed modules, a weekly audit and an overnight technical-debt report have nobody waiting, so they go to the Batches API.

SkillFitting 24 hours inside an SLA

When you submit a batch every few hours, the worst case is a document that arrives just after a submission. It waits a full interval for the next batch, and that batch may take the full 24 hours, so the worst case is the interval plus 24 hours. For the guide's 30-hour SLA, a submission every 4 hours gives a worst case of 28 hours and leaves 2 hours to validate and resubmit. Every 6 hours reaches exactly 30 hours, which meets the SLA only if nothing needs a second attempt.

Interactive · fitting a 24-hour window inside a 30-hour SLA

Worst-case time from document arrival to result for four batch submission intervals; each column stacks the wait for the next submission and the 24-hour processing window against a 30-hour SLA line HOURS SINCE THE DOCUMENT ARRIVED 0 h 6 h 12 h 18 h 24 h 30 h 36 h every 2 h batch ≤ 24 h worst case 26 h 4 H TO SPARE every 4 h wait 4 h batch ≤ 24 h worst case 28 h 2 H TO SPARE every 6 h wait 6 h batch ≤ 24 h worst case 30 h MEETS IT, 0 H SPARE every 12 h wait 12 h batch ≤ 24 h worst case 36 h 6 H LATE 30-HOUR SLA

Choose how often a batch is submitted, and see the worst case for a document that arrives just after a submission.

Every 2 hours: worst case 2 + 24 = 26 hours. Four hours inside the SLA, at the cost of many small batches.

Every 4 hours: worst case 4 + 24 = 28 hours. Inside the 30-hour SLA with two hours to handle results. This is the guide's example.

Every 6 hours: worst case 6 + 24 = 30 hours. It just meets the SLA, with no time left to validate results or resubmit a failure.

Every 12 hours: worst case 12 + 24 = 36 hours. A document arriving just after a submission misses the SLA by six hours.

Time reads downward from the document's arrival. Amber is the wait for the next submission, and blue is the 24-hour window, drawn at its maximum because nothing shorter is guaranteed. Green leaves time to handle results, amber meets the dashed SLA line with none, and red misses it.

KnowledgeOne request, one turn of your tools

Each request in a batch runs on its own, with none of your code listening. If Claude answers with a tool_use block for a tool you run, that request ends there, because running the tool and sending back a tool_result takes a new request. So the batch API does not support multi-turn tool calling within a single request: it cannot execute your tools mid-request and return results. A single-turn extraction fits a batch well, while an agent loop over your own tools belongs on the synchronous API.

More precise than the guide. A batch request cannot stop, wait for your code to run a tool, and continue. The docs still list tool use and multi-turn conversations as batchable, because a request can carry earlier turns and their tool results, and server tools such as web search run inside it (checked 2026-10-01). Exam answer: the batch API does not support multi-turn tool calling within a single request.

SkillResubmitting only what failed

Once a batch has ended, each request is succeeded, errored, expired or canceled, and only succeeded is billed. A batch of 10,000 invoices ends with 9,962 succeeded and 38 errored. Resending all 10,000 pays again for 9,962 results you already have, and 31 of the 38 fail again: their invalid_request_error says each document exceeded the context limit, and an invalid request must be changed before it is resent. The fix is to resubmit only the 38, found by custom_id, with the oversized documents chunked under new IDs and the 7 server errors resent unchanged. Expired requests are resubmitted too.

# one request per document
{"custom_id": "inv-20260921-00417",
 "params": {…, "tools": [extract_invoice],
            "messages": [{"role": "user", "content": "…document text…"}]}}
# after the batch ended: 9,962 succeeded, 38 errored
31 × invalid_request_error  — document exceeded the context limit
→ chunk into "inv-20260921-00417-p1" … "-p3", resubmit
 7 × api_error              — server error
→ resubmit unchanged, same custom_id

SkillRefine the prompt on a sample first

A batch reports its problems only after it ends, because even each request's params are validated asynchronously. A weak prompt therefore surfaces only after every document has cost money. Before batching the 10,000, run a sample of 50 documents that spans invoices, contracts and support emails through the synchronous API, and refine the prompt on what fails. Suppose 6 of the 50 put the PO number in invoice_number: one prompt fix and a second sample run take minutes, and the first-pass success rate rises before any batch is sent. Found after the batch, the same 12% error rate means about 1,200 resubmissions and another wait of up to 24 hours.

Ask one question: is a person or a pipeline blocked until this result arrives? Yes → synchronous API, whatever the cost. No → Message Batches API, submitted often enough that the interval, plus 24 hours, plus the time you need to handle results, fits inside the SLA.
Takeaway
  • Batch halves the price but promises only 24 hours, so it suits overnight reports, weekly audits and nightly test generation, never a pre-merge check.
  • Every 6 hours just meets a 30-hour SLA with no slack. The guide's answer is every 4 hours, which leaves 2 hours to handle results.
  • Match results by custom_id, and resubmit only the failures, changing any request that was invalid.

check yourself

0 / 3 answered

Contracts arrive around the clock and must reach the ledger within 36 hours of arriving. Extraction runs through the Message Batches API, and after each batch ends the team needs 3 hours to validate the results and route low-confidence records to human review.Which submission schedule meets the SLA with the fewest batches per day?

Select one.

A nightly batch of 8,000 support emails hit the 24-hour limit: 7,400 succeeded and 600 expired. The 600 emails arrived about 5 hours before the batch was submitted, and the SLA is 30 hours from arrival, so about one hour remains.What is the most effective way to handle the 600 expired requests?

Select one.

Your team will batch-process 40,000 archived contracts next week with a new extraction prompt. Last month a 5,000-document batch returned 900 records that failed schema validation, and fixing the prompt afterwards meant resubmitting all 900 and waiting another day.Which TWO steps best raise the first-pass success rate before the 40,000 are submitted?

Select two.

sources

  1. Claude Platform Docs, "Batch processing": all usage at 50% of standard prices; most batches finish within 1 hour; results when all requests complete or after 24 hours, and unprocessed requests expire; 100,000 requests or 256 MB per batch; results available for 29 days; custom_id of 1–64 letters, digits, hyphens and underscores; results in any order, matched by custom_id; the four result types and which are billed; fix an invalid_request_error before resending; asynchronous params validation and the advice to dry-run a request shape with the Messages API; tool use, server tools and multi-turn conversations in the "What can be batched" list. Checked 2026-10-01.
  2. Claude Code Docs, "Best practices", "Fan out across files": test on a few files, refine the prompt, then run on the full set. Checked 2026-10-01.
  3. Claude Platform Docs, "Handle tool calls": your code runs a client tool and returns the tool_result in a new request. Checked 2026-09-30.
  4. "Non-blocking, latency-tolerant" workloads, the pre-merge versus overnight split, the 4-hour / 30-hour SLA example, resubmission by custom_id with chunking, and sample-set prompt refinement: Claude Certified Architect — Foundations Exam Guide v1.0, Domain 4.5, Exercise 3 and Sample Question 11. The upload check, PR #812 and the document counts are this course's scenario details.