» Tag
evaluation
5 postsLLMs Are Honest in Prose but Hallucinate Under JSON Schemas
Study finds LLMs admit uncertainty in prose but fabricate data under required JSON schemas, with 10 of 13 models hallucinating 100% of the time.
TutorMoments: Testing If AI Tutors Know When to Help or Hold Back
Allen AI's TutorMoments benchmark tests whether LLM tutors know when to scaffold and when to push students toward harder reasoning.
Your AI Benchmark Might Be Measuring the Harness, Not the Model
Discover how software bugs can impact AI model evaluation. Understanding the role of the measurement system is crucial for assessing true model performance.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comLLM-Generated Tests Influence Implementation Selection Outcomes
Key findings on the impact of LLM-generated tests on implementation selection and evaluation.
Evaluating AI Agent Skill Performance with NVIDIA SkillEvaluator
NVIDIA SkillEvaluator is an open-source tool for measuring AI agent skill performance.