Task statement 2.4 · Output evaluation and validation
2.4 · When Human Review Is Required
In Wrenfield Housing, the course example, Claude drafts everything from internal brainstorms to figures for the regulator. The AI policy says a person checks anything that goes to tenants, the board or the regulator. It does not say how much checking each draft needs, so one draft nearly reached the regulator with only its writer's read-through.
share
The tempting habit is to give every draft the same check. A quick read for everything lets a wrong figure reach a regulator, because the writer reads for wording and cannot see the source system. Sign-off for everything turns reviewers into a queue, so they skim, and the drafts that matter get skimmed too. The other trap is trusting how sure the draft sounds. The Claude Help Center warns that Claude can write things that look correct but are very mistaken, so a confident tone tells you nothing about whether a check is needed.
KnowledgeWhat decides how much checking an output needs
Human review is a person reading an output before it is used or sent. Additional verification goes further: someone checks the output's claims against a source outside the conversation, such as the system a figure came from. Both cost time, so the amount should follow what is at stake. Anthropic's AI Fluency course says that whoever uses or shares an output takes responsibility for verifying and vouching for it, and that different contexts carry different expectations. It gives no fixed scale, so the questions below are common practice, not an Anthropic rule. They rule out any answer that chooses the check by the output's format, such as "it is only an email", or by how confident the draft sounds.
- Who receives it? Only you, your own team, or people outside it: customers, tenants, the board, a regulator or the public.
- What happens if it is wrong, and can it be undone? A team note can be corrected next week; a posted letter or a filed return cannot be called back.
- Is it regulated, legal, financial, or about a specific person? These outputs carry duties beyond being correct, and some need a qualified reviewer under Anthropic's own policy.
- Does it hold facts Claude could not know? Your organisation's figures that were never in the conversation, or events after the model's knowledge cutoff.
- Must an expert sign it off? Some outputs need a person with the knowledge and the authority to answer for every claim.
Interactive · four outputs, four levels of checking
Choose one of Wrenfield's outputs and see which checks it needs before it is used or sent.
Brainstorm: only the writer reads it and nothing acts on it, so it needs no review. A bad idea is simply not chosen.
Team update: the operations team acts on it, but can correct it next week, so the writer's self-check against the export is enough.
Tenant letter: it reaches 1,200 households and cannot be called back, so a second person, the communications lead, reads it first.
Regulator return: it is regulated, goes on the record and holds figures Claude could not check, so a second check against the repairs system and the director's sign-off come first.
The fourth question needs the most care, because no self-check can answer it. Claude knows only what the conversation and its training give it, and each model has a knowledge cutoff after which it may not know about events. When a draft states a figure from your own systems, or a recent fact, rereading it, by you or by Claude, checks the wording, not the fact. Only the source can confirm the claim, and how to check against a source is the subject of 2.3.
KnowledgeThree kinds of check
A self-check is the writer reading the draft against their notes and criteria. Anthropic's prompting guide adds a cheap step: ask Claude to verify its answer against stated criteria before it finishes. A self-check catches slips, but it shares the writer's blind spots and Claude's. A second person's review is a colleague who did not write the draft reading it as its reader will, so they see gaps the writer no longer can. Expert verification is a person with the training and authority in the field checking the claims against the original source and signing the result. Each kind adds to the ones before it:
The writer, against the notes and criteria, plus "verify your answer against these criteria" in the prompt.
MissesAnything the writer and Claude both got wrong, and any fact neither can see.
A colleague who did not write it reads it as the recipient will.
MissesA wrong figure that reads well, unless the reviewer opens the source.
A qualified person checks each claim against the original source and signs off.
CostsThe most time, so keep it for high-stakes outputs.
SkillWorked example: the covering note that was nearly sent
Each quarter Wrenfield sends the regulator a data return with a short covering note. In October an operations officer asked Claude to draft the note, with last quarter's note as the example and the figures attached. The officer saw the result as an email rather than part of a return, so they read it once and queued it to send:
# before: sorted by format Output: an email to the regulator → self-check, then send → "All damp and mould reports are inspected within 10 working days." # after: sorted by what is at stake Receives it: the regulator. Undone? No, it goes on the record. Regulated: yes. Facts Claude could not know: inspection times. → Level 3: second check against the repairs system; director signs → the system shows a median of 15 working days; the claim is cut
The cause was sorting the draft by its format. The inspection sentence was in no attachment, and the backlog export has no inspection field, so Claude wrote a plausible line in the style of last quarter's note. The officer's read-through could not catch it, because the sentence read well and no source was open. The operations director, copied in, stopped the email. Wrenfield now sorts anything sent to the regulator as level 3, whatever its format.
Anthropic's rule, not only practice. Anthropic's Usage Policy, the rules for using its products, lists high-risk areas such as housing, legal, finance and employment. When Claude's output is a recommendation about an individual in those areas, a qualified person must review it before it is delivered or acted on. A new version of the policy takes effect on 12 November 2026, and both versions keep this requirement (checked 2026-10-15).
| Only you, and nothing acts on it | No review: use it as it is. |
| Your team, and it can be corrected | Self-check against your notes and criteria. |
| People outside the team, and it cannot be called back | Add a second person's review before it goes. |
| Regulated, legal, financial or about a person, or built on facts Claude could not know | Add expert verification against the source, with sign-off. |
- Choose the check by the stakes, not by the format of the output or how confident Claude sounds.
- Rereading a draft, by you or by Claude, checks the wording; only the source can confirm a figure Claude could not see.
- Self-check, second person's review and expert verification add to each other; a higher level never skips a lower one.