» Tag
data-contamination
1 postsHow a shared JSONL file inflated our eval score to 0.94
A retrieval-based few-shot selector shared its index with the eval set, inflating accuracy from 0.79 to a fake 0.94. Here's the contamination check that catches it.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com