Review-Loop Engineering: Designing Real Human Oversight for Agent Loops
Why 'human in the loop' isn't real oversight for AI coding agents, and how review-loop engineering designs packets that expose gaps, not just green checks.
Running coding agents as scheduled loops — wake, find work, act, test, hand off — is becoming standard practice. But most engineering attention still goes to the machine side: how many subagents spawn, which model judges completion, what the token budget is. The human side is often reduced to a single 'approve' button reviewing a stack of opaque diffs, which functions more like an inbox than actual oversight.
This piece names the missing layer 'review-loop engineering': designing the review packet, escalation rules, and feedback path so the accountable engineer can actually understand and redirect an agent's work. It proposes three distinct loops — the agent loop (makes the change), the verifier loop (tests, scores, or challenges it), and the human loop (sets direction, decides escalations, turns findings into better specs and guardrails). The first two can run fast; the human loop is inherently slow because comprehension takes time.
The core argument: 'human in the loop' is not by itself a safety design. A person can be technically present yet functionally absent when handed an unfamiliar diff and a green checkmark. Verifiers — tests, a second skeptical agent, invariants — are necessary but not sufficient, since any check only sees what its rubric can ask. The practical takeaway for engineering teams is to stop optimizing agent summaries for reassurance and instead build review artifacts that clearly state what changed, what was proven, what wasn't, and what decision is actually left to the human.