» Tag
llm-as-judge
4 postsThe 75% False-Negative Wall LLM Reviewers Can't Move
Why LLM-based reviewers hit a fixed 75% false-negative rate, and why voting, reruns, and prompt calibration all fail to move it.
Evaluation Debt: Why Offline Agent Evals Fail in Production
Why offline eval frameworks miss real production failures in AI agents, and why session-based observability is replacing static test suites.
Mechanistic View Reveals How Bias Lives Inside LLM Judges
Study shows LLM-as-judge bias is encoded in activation geometry, enabling causal steering and better failure prediction than text-based methods.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comWhy LLM-as-judge scores flip between identical runs
An LLM judge gate flipped between 0.79 and 0.82 on identical inputs. Sampling temperature, model drift and vague rubrics cause the jitter; k-sampling and a noise band fix it.