Task statement 5.2 · Context management and reliability

Design effective escalation and ambiguity resolution patterns

The support agent resolves 55% of contacts on first contact, against the 80% target. Its logs show it calling escalate_to_human for standard damage replacements that arrive with photo evidence, while it handles requests that need a policy exception by itself. It escalates the wrong cases because nobody told it which ones are right.

Why it's tested

The tempting fixes each swap a decision rule for something easier to measure. A sentiment threshold sends the angry customer with a broken mug to a human, although policy covers the replacement. Looking up the order "to be helpful" before honouring a request for a person overrides the customer's choice. Choosing the account with the most recent order when two customers match is a guess, and a wrong guess refunds someone else.

KnowledgeFour checks, asked in order

An escalation trigger is a fact about the case that means a human should take it, and the guide names three: the customer asks for a human, policy has an exception or a gap, and the agent cannot make meaningful progress. A policy gap is a request the written policy is ambiguous or silent about. If policy refunds the difference when your own site's price drops within 14 days, a request to match a competitor's price $30 lower is not covered, so declining and approving would both invent policy. Complexity alone is not a trigger, because a complicated case that policy covers is the agent's job. One more check is not an escalation: when get_customer returns multiple customer matches, the agent asks for additional identifiers instead of selecting by heuristic. The checks run in order, as a ladder.

  1. Asked for a human? Escalate now, without investigating first.
  2. More than one get_customer match? Ask for another identifier.
  3. Policy silent or ambiguous on this request? Escalate.
  4. No meaningful progress possible? Escalate with what was tried.
  5. Otherwise resolve, acknowledging any frustration.

Interactive · where each case leaves the ladder

A top-to-bottom escalation ladder of four questions: explicit request for a human, multiple customer matches, policy coverage, and meaningful progress, each with an outcome to its right, ending in autonomous resolution INCOMING TURN — choose a case above — “I don’t want a bot. Just get me a person.” “Refund my last order.” get_customer returns two accounts named Jane Park. “Another site sells it $30 cheaper. Match it.” Policy covers own-site price drops only. “The mug arrived smashed. Photo attached. This is ridiculous.” process_refund fails twice with a processor error; there is no alternative payment route. check 1 Has the customer asked for a human? yes escalate_to_human now no investigation first no More than one customer match? yes ask for another identifier email or order number, never a guess no Does policy cover this exact request? not covered escalate_to_human policy gap · structured handoff covered Can the agent make meaningful progress? no escalate_to_human handoff includes what was tried yes resolve autonomously acknowledge frustration, offer the fix · escalate only if the customer asks again

Choose an incoming case, and see which question decides it.

Explicit request → escalate now. The agent does not investigate first, even if the issue looks easy.

Two matches → ask for an email address or order number. Picking the more recent account is a guess about identity.

Policy is silent on competitor pricing, so this is a gap → escalate with a structured handoff. Declining or approving would both invent policy.

Covered by policy, and the agent can act → resolve. Acknowledge the frustration, offer the replacement. If they then ask for a person, the first question applies.

No meaningful progress is possible → escalate, with what was attempted in the handoff.

Time reads downward. The order is the point: an explicit request for a human is honoured before anything else is checked. Nothing on the ladder reads the customer's sentiment or the model's confidence.

KnowledgeA request for a person versus frustration

The guide separates two customers who can sound alike, and the difference is what they asked for. A customer who explicitly demands a human gets escalate_to_human immediately, without first attempting investigation, even when the issue looks easy, because looking up the order first replaces the customer's stated choice with the agent's. A customer who is only frustrated is different. When the issue is straightforward and within the agent's capability, the agent acknowledges the frustration and offers to resolve it, escalating only if the customer reiterates that they want a person.

Explicit request

"I don't want a bot. Get me a person."

The issue is a $24 late-delivery refund the agent could issue.

Next move: escalate now, passing what the customer has already said.

Frustration only

"Third time I'm writing. This is ridiculous."

The same $24 refund, and no request for a person.

Next move: acknowledge it, offer the refund, escalate if they ask again.

KnowledgeWhy sentiment and confidence are poor proxies

Sentiment-based escalation and self-reported confidence scores both look like measurements of difficulty, but neither measures actual case complexity. Sentiment measures mood, so the angry customer with a photographed broken mug is escalated while the calm customer asking for a competitor price match is not. A self-reported confidence score from 1 to 10 is the model's opinion of itself, and this agent's opinion is already wrong. It escalates easy cases, which shows its confidence is low where it should be high and high where it should be low, so a threshold on that score routes the same mistakes. Only a score calibrated against labelled outcomes can route work, as 5.5 shows for human review.

SkillCriteria plus examples, in the system prompt

Take the guide's case. The symptom is 55% first-contact resolution, with photo-backed damage replacements escalated and policy exceptions handled alone. The cause is a system prompt that says only "Escalate complex or sensitive cases", so the agent draws its own boundary by how each case feels. The fix is explicit escalation criteria with few-shot examples demonstrating when to escalate versus resolve autonomously. This is the proportionate first response, because it repairs the unclear boundary directly, before anyone builds a classifier that needs labelled tickets and infrastructure. The examples carry the reasoning to cases the criteria do not list, and Anthropic's prompting guide recommends three to five diverse ones in <example> tags.

Escalate complex or sensitive cases.   ← before
## When to escalate                     ← after
Escalate when the customer asks for a person, when policy is silent
or needs an exception, or when you cannot make progress.
Do not escalate because a case is complicated or a customer is upset.
<example> "The mug arrived smashed. Photo attached. Ridiculous."
Resolve: damage with a photo is standard policy. Acknowledge, replace.
</example>
<example> "Another site has this kettle $30 cheaper. Match it."
Escalate: policy covers our own price drops only. This is a gap.
</example>
<example> get_customer returns two accounts named Jane Park.
Ask for the account email or order number. Never pick one.
</example>

The third example covers ambiguity, not escalation. Jane Park asks for an $84 refund, and get_customer returns accounts C-1042 and C-2210 under her name. Picking the one with the most recent order is heuristic selection, and a wrong guess sends the $84 through process_refund on someone else's account. One more identifier settles the match in a single turn. Anything the agent does escalate needs a structured handoff, because the human cannot see the transcript: customer ID, root cause, amount and recommended action, as 1.4 covers.

Ask in this order: did the customer ask for a human? Is there more than one account match? Does policy cover this request? Can the agent make progress?
Asked for a human, policy silent or needs an exception, or no progress possibleCall escalate_to_human with a structured handoff. For a request, do it before any investigation.
More than one account matchAsk for another identifier, then run the checks again.
None of theseResolve it. Acknowledge any frustration, and escalate only if the customer asks for a person.
Takeaway
  • "Get me a person" is honoured at once; "this is ridiculous" gets an apology and an offer to fix it.
  • Anger and a model's opinion of itself do not measure how hard a case is.
  • The first fix for escalating the wrong cases is written criteria with few-shot examples, not a classifier.

check yourself

0 / 3 answered

A customer opens a chat: "I don't want a bot. Put me through to a person." The same message gives an order number and asks to change the shipping address on that unshipped order, which the agent can do with one tool call. First-contact resolution is 72% against the 80% target.What is the most appropriate first step for the agent?

Select one.

Last week's logs show two problems. The agent refunded a $60 order to a gift card on its own, although policy only describes refunds to the original payment method. It also escalated four customers who wrote in capitals about damaged items with photos, which policy covers.Which TWO additions to the system prompt most directly correct both behaviours?

Select two.

A customer in live chat asks to cancel a $120 annual subscription and get a refund. get_customer, searched by phone number, returns two accounts, Sam Ortiz and Dana Ortiz, who share a household phone. Both have an active subscription. The customer is still in the chat.Which is the most proportionate next step?

Select one.

sources

  1. Claude Platform Docs, "Prompting best practices", "Use examples effectively" and "Add context to improve performance": few-shot examples improve accuracy and consistency; make them relevant and diverse, wrap them in <example> tags, include 3–5; explain why a behaviour matters. Checked 2026-10-01.
  2. Anthropic Engineering, "Building Effective Agents": agents "can then pause for human feedback at checkpoints or when encountering blockers"; in customer support, "success can be clearly measured through user-defined resolutions". Checked 2026-10-01.
  3. Anthropic's documentation has no escalation-criteria or confidence-calibration guidance of its own. The three triggers, the explicit-request versus frustration distinction, the multiple-match rule, the competitor price-matching example and the 55%-versus-80% case: Claude Certified Architect — Foundations Exam Guide v1.0, Domain 5.2 and Sample Question 3. The 14-day own-site price rule, the dollar amounts and account IDs are this course's scenario details.