Favur Evals: a public benchmark for which AI model codes best
Favur Evals is a public, vendor-independent leaderboard comparing AI models on real software engineering tasks across eight measurable dimensions.
Favur has launched Favur Evals, a public leaderboard that runs the same software task through a full team of AI agents — planner, architect, tester, coder, reviewer, builder — while swapping only the underlying model each time. The goal: give engineers a concrete answer to which model to trust for a given coding job, instead of relying on vendor claims.
The results resist a single winner. The current top run is a mix of Gemini Flash 3.5 and Gemini Flash Lite across different agent roles, while an all-OpenAI run produced the strongest test suites, all-DeepSeek won on cost efficiency, and all-Qwen delivered the cleanest zero-failure run. Every result breaks down into eight engineering subjects — code quality, test quality, cost efficiency, velocity, tool discipline, effort efficiency, process discipline, and deliverables — plus behavioral fingerprints like cache usage, reasoning depth, and tool cadence.
Crucially, none of the scoring relies on an LLM judge: every metric is recomputed from artifacts the run itself produces, such as lint output, complexity metrics, and test results. The rubric is public and versioned, and scores expand into their formulas on click. Run independently without vendor sponsorship, Favur Evals gives engineering teams a transparent, reproducible way to pick models by the actual job they're hiring for.