» Tag
data-quality
5 postsIf 30% of SWE-Bench Pro Tasks Are Broken, Add an Uncertainty Budget
OpenAI's SWE-Bench Pro audit found ~30% of tasks broken. Learn how to version task validity and report uncertainty intervals in benchmarks.
When an LLM's Training Data Expires: Fixing a Car Pricing Engine
An AI car-pricing engine broke because its LLM recalled stale exchange rates, revealing risks of mandatory retrieval and cache provenance in LLM apps.
Generating the database from the answer key in text-to-SQL benchmarks
A UIUC audit found over half of BIRD and Spider 2.0 annotations wrong. One developer inverted the process: declare the answer first, then generate a database that satisfies it.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comAI Technical Debt: Why Generated Code Still Needs Metadata and Review
AI-generated code speeds up data engineering but hides risk. Without structured metadata, review and governance, teams accumulate hidden intent debt.
Maintaining Match Confidence on the Graph Edge: The Cost of Discarding Splink Scores
Discarding match probabilities impacts the accuracy of knowledge graphs. Retaining them as edge properties improves system reliability.