Reading Agent Traces Reveals What Evals Can't Decide

While building a lightweight conference schedule assistant for AI Engineer World's Fair, an engineer set up model routing that gave cheap open-weight models to anonymous users and stronger models to signed-in ones. When asked "who are some famous speakers," the agent's router sent the query to a path with only one tool, getTracks, which returns track metadata, not speaker names. The model still confidently listed real and entirely fabricated names, and when challenged, it doubled down, falsely claiming the data came straight from the API. Because the trace showed "Used Tools: getTracks," the hallucination looked grounded until someone actually opened the tool output.

The piece argues that while a groundedness eval could have caught this specific failure, the instinct to write that eval in the first place only comes from reading real traces. Instead of trying to test an open-ended universe of user phrasing, the author recommends targeting the agent's defined intents plus deliberate out-of-data cases—like querying an empty speakers table—using small, deterministic checks that don't require an LLM judge.

The more important point is about decisions evals can't make: knowing whether a cheaper, lower-scoring model is acceptable for a given audience is a judgment call, not a test result. Faced with several competing fixes—upgrading the free tier, fixing routing, tightening the prompt, adding a groundedness eval, or simply accepting the risk—the author says only reading enough traces gave the confidence to choose.

Finally, the post highlights practical observability: logging agent inputs, outputs, and tool calls in Sentry with a stable conversation_id lets engineers find traces alongside errors, performance data, and logs from the same request—something a standalone eval score can never show on its own.