Evaluation Debt: Why Offline Agent Evals Fail in Production
Why offline eval frameworks miss real production failures in AI agents, and why session-based observability is replacing static test suites.
Thirty-eight percent of AI teams now cite evaluation as their primary blocker. The issue isn't a weak framework choice — LangSmith, Braintrust, Phoenix, and similar tools all score agents against a fixed held-out test set, but that set is a snapshot of last week's world. Production traffic drifts past it almost immediately, so offline evals measure the past while real usage tests the future.
Multi-agent systems make this worse: five agents mean 2-5x more execution paths, and ten agents produce emergent failures no single-agent eval can catch. Teams commonly end up stitching together four separate tools — offline trajectory eval, gateway checks, guardrails, and business-logic validation. LLM-as-judge scoring, widely adopted to avoid manual labeling, carries systematic weaknesses too: position and length bias, error rates above 50% on complex tasks, and only 64-68% agreement with domain experts — meaning teams gating CI on these scores are accepting a quiet, measurable risk.
The teams pulling ahead aren't picking better eval frameworks; they're building infrastructure that treats agent sessions as first-class data. Every turn — reasoning, tool calls, cost, latency — gets logged and queried in one place, with evals run live against that session data rather than a hand-assembled dataset. Labeled production turns then feed back into training and reward signals, closing the loop between what actually fails and what the system learns next — the difference between agents that keep improving and agents that plateau.