» Tag
software-testing
9 postsThe 75% False-Negative Wall LLM Reviewers Can't Move
Why LLM-based reviewers hit a fixed 75% false-negative rate, and why voting, reruns, and prompt calibration all fail to move it.
AI Coding Agents Are Breaking Diff-Based Code Review
AI coding agents make diff-only code review unreliable; intent now lives in test names, builder APIs, and contract tests instead.
When Green Tests Lie: Ten Patterns Behind False Positives
Ten real examples from a single audit day show how passing tests and checks can mask broken software, and how to actually test the checkers themselves.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comTesting AI Agents: The Bug Hides in the Answer, Not the Trace
An AI agent passed eight straight safety tests, then silently broke one - a case study in why traces, not answers, reveal agent failures.
Bun's Million-Line AI Rewrite Passed All Tests, Still Shipped 19 Bugs
Bun's AI-driven Zig-to-Rust rewrite passed every test yet shipped 19 regressions, exposing the gap between test coverage and true verification.
The AI Agent Hunting Bugs in My App While I Sleep
One AI agent hunts bugs in a live app hourly while another reproduces and fixes them, using confidence scoring and reproduction rules to stay trustworthy.
Lessons from a Week Evaluating an AI PR Reviewer
Lessons from evaluating an AI PR review plugin: fixing the wrong skill, why risk classification matters, and how upstream evidence quality shapes review accuracy.
Evaluating Agentic Autofix: Canary Gates and Failure Taxonomies
GitHub's July 2026 agentic autofix preview prompts a proposed framework for evaluating AI security patches via canary testing and failure classes.
Give the AI Agent the Harder Job: Ambiguity Yields Better Results
Giving AI agents ambiguous, harder tasks—paired with rigorous verification—surfaces hidden bugs and tackles the backlog work engineers never had time for.