Task statement 4.6 · Prompt engineering and structured output
Design multi-instance and multi-pass review architectures
A CI job asks Claude to fix token refresh across three files in src/auth/, then asks the same session to review its diff. The review reports no issues, and the fix ships with a race between refresh and logout that the generating session had considered and judged acceptable.
share
The tempting fixes keep the reviewer inside the generator's context. Adding "review critically" to the same session, or turning on extended thinking, looks rigorous, but the session still holds the reasoning that excused the race, so it reaches the same verdict. The second trap is scale: one pass over a 14-file pull request gives some files careful attention and others almost none.
KnowledgeWhy self-review misses what the author already excused
While Claude generates code, its context window fills with reasoning context: the options it weighed, the assumptions it made and the risks it accepted. When the same session reviews the result, that reasoning is still there, so the model is less likely to question its own decisions in the same session. In the token-refresh job, "refresh can overlap logout, acceptable, the window is milliseconds" is already in context, so the race reads as settled rather than as a finding. A self-review instruction or extended thinking adds effort inside that same context, which is why the guide ranks both below an independent review instance for catching subtle issues.
The fix is a second independent Claude instance: a new invocation with its own context window, given the diff and the review criteria but none of the generator's reasoning. Claude Code's docs say a reviewer in a fresh context "sees only the diff and the criteria you give it, not the reasoning that produced the change". In CI that means a separate claude -p call or a subagent, not another prompt in the generating session.
- The generating session writes the diff and keeps its reasoning.
- Only the diff and the criteria go to new instances.
- Each changed file gets its own local pass.
- An integration pass reads the data flow between files.
- Each finding's self-reported confidence decides where it goes.
Interactive · who reviews, and with what in context
Choose who reviews the generated code.
Self-review: the reviewer shares the generator's context window, reasoning included. The race was ruled acceptable before the review began, so it is not questioned.
Independent, multi-pass: only the diff and the criteria cross the border into each second independent Claude instance. Per-file passes give even depth, the integration pass sees the refresh/logout interaction, and confidence routes each finding.
refresh.ts and logout.ts both write the session, because each local pass sees one file. Confidence is the model's own, from 0 to 1.KnowledgeThree ways to "review it again", and which one works
The three options differ only in what the reviewer's context holds.
"Review your changes for bugs", sent to the session that wrote the code.
Why it missesThe reasoning behind each decision is still in context, so the decisions are re-read as correct.
More reasoning tokens on the review step, in the same session.
Why it missesLonger thinking starts from the same assumptions, so the excused race stays excused.
A new instance given the diff and the criteria only.
Why it worksIt sees only the code, not the author's intentions. That is why teams don't let authors approve their own pull requests.
A self-check is not an independent review. Anthropic's prompting guidance suggests ending a prompt with "Before you finish, verify your answer against [test criteria]". That finds mistakes visible against explicit criteria, not a decision the model already judged correct. The same guidance runs its self-correction chain (draft, review, refine) as separate API calls.
| Yes: the same session, a resumed or forked copy, or an added self-review or thinking step | It is self-review. Move it to a second independent Claude instance given only the diff and the criteria. |
| No: a new instance with only the diff and the criteria | It is an independent review. Keep it that way. |
SkillSplitting a 14-file review into passes
Independence decides who reviews, and the number of passes decides how much attention each file gets. Suppose a pull request changes 14 files across the stock tracking module and one pass reviews them all. Attention dilution means the model's attention is divided across all 14 files at once, so some files get detailed feedback and others superficial comments, obvious bugs are missed, and a pattern flagged in one file is approved in another. A larger context window lets all 14 files fit, but attention is still divided across them. One focused local pass per file gives each file a whole pass, and a separate integration pass then examines the cross-file data flow that no single-file pass can see. Every pass still runs without the generator's reasoning, because multi-pass and multi-instance are separate fixes.
One pass over 14 files
review(all 14 files) → stock.ts: 9 detailed findings → reorder.ts: "looks fine" → unchecked null flagged in sku.ts, same code approved in batch.ts
Result: uneven depth and contradictory findings.
14 local passes, then 1 integration pass
review(stock.ts) … review(batch.ts) → same pattern, same verdict per file integrate(diff + local findings) → reorder.ts reads qty_on_hand, which sku.ts renamed
Result: even depth, plus the cross-file bug.
SkillConfidence on every finding, for routing
In a verification pass the model self-reports a confidence alongside each finding, so the pipeline can route findings instead of posting all of them. Findings above a threshold become PR comments, and the rest go to a further check or a person. A self-reported score is safe to route on only once it is calibrated, meaning checked against past findings labelled real or false, because only then do you know whether 0.8 means right four times in five. Domain 5 covers building that labelled set.
{"file": "src/auth/refresh.ts", "line": 42,
"issue": "writes session after logout cleared it",
"confidence": 0.9} // ≥ 0.8 → post on the PR
{"file": "src/auth/session.ts", "line": 7,
"issue": "helper name breaks convention",
"confidence": 0.4} // < 0.8 → check first
Anti-patternReviews that look independent and are not
- Resuming or forking the generating session to review. Both start from a copy of its history, so the reasoning comes back with it.
- Running three full-PR passes and posting only findings two of them agree on. A real bug that a pass catches only intermittently is voted out.
- The generating session keeps its reasoning, so its review re-approves its own decisions, however the review is worded.
- For subtle issues, a fresh instance beats self-review instructions and extended thinking.
- Large reviews split into per-file passes plus an integration pass, and calibrated confidence decides what gets posted.