Item 21: Verify what the agent did, not what it said it did

Verified with Claude Code 2.1.153
Stability: stable
Status: current

Why this matters

A subagent ends its run by composing a summary message back to the parent. That summary is the agent’s report of what it intended to do — generated by the same model that made the decisions, with all the same blind spots. It is not the diff. It is not the test output. It is not the file the agent claims to have written. It is one model’s account of its own work, and like any self-report it can be wrong in the specific way that matters most: the agent may believe it did the thing without actually having done it.

The failure modes are concrete. An agent says “implemented and tested” but never ran the tests. An agent says “fixed the bug in auth.ts:42” but the edit landed on the wrong line. An agent says “no call sites remain” but it searched with a typo. An agent says “all green” because it ran the wrong command. These aren’t lying agents — they’re agents whose narrative drifted from reality, which is exactly what one model writing about its own work will produce sometimes.

The discipline is to treat the summary as a claim and verify against the ground truth. For agents that touched code: git diff or git status before accepting the work. For agents that report findings: open one or two of the files they cited and confirm the finding exists. For agents that ran tests: see the actual test runner output, not the agent’s summary of it. This costs seconds and catches the cases where the agent’s confidence outran its actual work.

What to avoid

Approving “implemented the feature, all tests pass” without looking at what changed. Citing an agent’s findings to the user without spot-checking. Trusting an agent’s claim about a file you can read in two seconds. Composing the next agent’s prompt as if the previous one’s report is established fact rather than a claim.

What to do instead

For code-writing agents, the verification step is git diff (or git status to scan the surface), reviewed in the parent thread before reporting completion to the user. For research and audit agents, open one cited file at random and confirm the finding. For agents that run tests or builds, check the actual output. The cost is small; the cost of compounding an unverified agent claim into the next decision is large.

Phrase reports to the user in a way that doesn’t launder the agent’s claims into your own. “The agent reports all tests pass — confirmed by re-running” is honest; “all tests pass” without verification is borrowing the agent’s confidence.

Example

Verifying a code-writing agent’s claim.

[agent returns: "Migrated all three call sites to the new helper.
                Removed the legacy import. Tests pass."]

> git diff --stat
src/api/users.ts        | 12 ++++--------
src/api/orders.ts       |  8 +++-----
src/api/products.ts     | 10 ++++------
src/legacy-import.ts    |  4 ++--

[four files changed, not three — the agent edited legacy-import.ts
 too. Investigate before reporting "done" to the user.]

Verifying a research agent’s claim.

[agent returns: "All 17 call sites of `legacyAuthorize` are in
                src/auth/ and trivially migratable."]

> grep -rn legacyAuthorize src/ | wc -l
23

[agent missed six call sites — likely searched with case-sensitive
 grep when some are `LegacyAuthorize`. Re-run with the right query
 before acting on the report.]

The verification step is small. It’s also the difference between a session that compounds toward the user’s goal and one that compounds toward a confident-sounding wrong answer.

Things to Remember

An agent’s return message describes intent — it is not evidence of outcome
For code-writing agents, read the diff before accepting their summary
For research agents, spot-check by opening one or two of the files they cite
Treat ‘all tests pass’ from an agent as a claim until you see the test runner output