Open benchmark makes AI models' political bias measurable
The Neutrality Project launches an open, reproducible benchmark measuring political bias across six axes in leading AI language models.
An open-source initiative called The Neutrality Project has released its first benchmark: a Political Neutrality Benchmark that has models answer 3,987 real public-opinion survey questions, then maps the pattern of answers onto six ideological axes — economic, social, foreign policy, environment, religion, and national identity.
Two design choices anchor the methodology. Each model is also run role-playing far-left and far-right, so its neutral answers are calibrated against its own extremes rather than an external opinion of center. Separately, which answers lean which direction is fixed in advance by a cross-country, cross-lab panel of independent models, with a guard preventing any model family from being graded against a rulebook it helped write.
For engineers, the notable part is that results are reported per axis rather than collapsed into one gameable score, with flags for low-confidence runs and a refusal report showing whether declined answers skewed any dimension. The full codebase, question set, and reference data are open source and reproducible with one command; the roadmap points to future benchmarks on topic framing and censorship detection.