LLM-as-a-Verifier Turns Verification Into a New Scaling Axis
New research scales LLM verification without extra training, introducing continuous scoring that hits state-of-the-art on SWE-Bench, Terminal-Bench and more.
LLM-as-a-Verifier proposes verification—an LLM's ability to judge solution correctness—as a new scaling axis alongside pre-training, post-training, and test-time compute. Instead of prompting models for discrete scores like typical LM judges, it computes the expectation over scoring-token logit distributions to produce continuous scores, enabling scaling across score granularity, repeated evaluation, and criteria decomposition.
The authors show finer score granularity improves separation between correct and incorrect solutions, while repeated evaluation and criteria decomposition reduce variance and complexity to boost accuracy further. A cost-efficient ranking algorithm built on these continuous scores helps select the best candidate solution. Without any additional training, the framework achieves state-of-the-art results on demanding agentic benchmarks: Terminal-Bench V2, SWE-Bench Verified, RoboRewardBench, and MedAgentBench.
For engineers, the fine-grained verification signal doubles as a proxy for task progress, and the team built a Claude Code extension so developers can monitor and improve their own agentic systems. The same dense feedback also improves sample efficiency for RL methods like SAC and GRPO on robotics and math reasoning tasks, making this a practical, training-free tool for building and debugging agent pipelines.