Task statement 5.2 · Context management and reliability
Design effective escalation and ambiguity resolution patterns
The support agent resolves 55% of contacts on first contact, against the 80% target. Its logs show it calling escalate_to_human for standard damage replacements that arrive with photo evidence, while it handles requests that need a policy exception by itself. It escalates the wrong cases because nobody told it which ones are right.
share
The tempting fixes each swap a decision rule for something easier to measure. A sentiment threshold sends the angry customer with a broken mug to a human, although policy covers the replacement. Looking up the order "to be helpful" before honouring a request for a person overrides the customer's choice. Choosing the account with the most recent order when two customers match is a guess, and a wrong guess refunds someone else.
KnowledgeFour checks, asked in order
An escalation trigger is a fact about the case that means a human should take it, and the guide names three: the customer asks for a human, policy has an exception or a gap, and the agent cannot make meaningful progress. A policy gap is a request the written policy is ambiguous or silent about. If policy refunds the difference when your own site's price drops within 14 days, a request to match a competitor's price $30 lower is not covered, so declining and approving would both invent policy. Complexity alone is not a trigger, because a complicated case that policy covers is the agent's job. One more check is not an escalation: when get_customer returns multiple customer matches, the agent asks for additional identifiers instead of selecting by heuristic. The checks run in order, as a ladder.
- Asked for a human? Escalate now, without investigating first.
- More than one
get_customermatch? Ask for another identifier. - Policy silent or ambiguous on this request? Escalate.
- No meaningful progress possible? Escalate with what was tried.
- Otherwise resolve, acknowledging any frustration.
Interactive · where each case leaves the ladder
Choose an incoming case, and see which question decides it.
Explicit request → escalate now. The agent does not investigate first, even if the issue looks easy.
Two matches → ask for an email address or order number. Picking the more recent account is a guess about identity.
Policy is silent on competitor pricing, so this is a gap → escalate with a structured handoff. Declining or approving would both invent policy.
Covered by policy, and the agent can act → resolve. Acknowledge the frustration, offer the replacement. If they then ask for a person, the first question applies.
No meaningful progress is possible → escalate, with what was attempted in the handoff.
KnowledgeA request for a person versus frustration
The guide separates two customers who can sound alike, and the difference is what they asked for. A customer who explicitly demands a human gets escalate_to_human immediately, without first attempting investigation, even when the issue looks easy, because looking up the order first replaces the customer's stated choice with the agent's. A customer who is only frustrated is different. When the issue is straightforward and within the agent's capability, the agent acknowledges the frustration and offers to resolve it, escalating only if the customer reiterates that they want a person.
"I don't want a bot. Get me a person."
The issue is a $24 late-delivery refund the agent could issue.
Next move: escalate now, passing what the customer has already said.
"Third time I'm writing. This is ridiculous."
The same $24 refund, and no request for a person.
Next move: acknowledge it, offer the refund, escalate if they ask again.
KnowledgeWhy sentiment and confidence are poor proxies
Sentiment-based escalation and self-reported confidence scores both look like measurements of difficulty, but neither measures actual case complexity. Sentiment measures mood, so the angry customer with a photographed broken mug is escalated while the calm customer asking for a competitor price match is not. A self-reported confidence score from 1 to 10 is the model's opinion of itself, and this agent's opinion is already wrong. It escalates easy cases, which shows its confidence is low where it should be high and high where it should be low, so a threshold on that score routes the same mistakes. Only a score calibrated against labelled outcomes can route work, as 5.5 shows for human review.
SkillCriteria plus examples, in the system prompt
Take the guide's case. The symptom is 55% first-contact resolution, with photo-backed damage replacements escalated and policy exceptions handled alone. The cause is a system prompt that says only "Escalate complex or sensitive cases", so the agent draws its own boundary by how each case feels. The fix is explicit escalation criteria with few-shot examples demonstrating when to escalate versus resolve autonomously. This is the proportionate first response, because it repairs the unclear boundary directly, before anyone builds a classifier that needs labelled tickets and infrastructure. The examples carry the reasoning to cases the criteria do not list, and Anthropic's prompting guide recommends three to five diverse ones in <example> tags.
Escalate complex or sensitive cases. ← before ## When to escalate ← after Escalate when the customer asks for a person, when policy is silent or needs an exception, or when you cannot make progress. Do not escalate because a case is complicated or a customer is upset. <example> "The mug arrived smashed. Photo attached. Ridiculous." Resolve: damage with a photo is standard policy. Acknowledge, replace. </example> <example> "Another site has this kettle $30 cheaper. Match it." Escalate: policy covers our own price drops only. This is a gap. </example> <example> get_customer returns two accounts named Jane Park. Ask for the account email or order number. Never pick one. </example>
The third example covers ambiguity, not escalation. Jane Park asks for an $84 refund, and get_customer returns accounts C-1042 and C-2210 under her name. Picking the one with the most recent order is heuristic selection, and a wrong guess sends the $84 through process_refund on someone else's account. One more identifier settles the match in a single turn. Anything the agent does escalate needs a structured handoff, because the human cannot see the transcript: customer ID, root cause, amount and recommended action, as 1.4 covers.
| Asked for a human, policy silent or needs an exception, or no progress possible | Call escalate_to_human with a structured handoff. For a request, do it before any investigation. |
| More than one account match | Ask for another identifier, then run the checks again. |
| None of these | Resolve it. Acknowledge any frustration, and escalate only if the customer asks for a person. |
- "Get me a person" is honoured at once; "this is ridiculous" gets an apology and an offer to fix it.
- Anger and a model's opinion of itself do not measure how hard a case is.
- The first fix for escalating the wrong cases is written criteria with few-shot examples, not a classifier.