Testing LLMs Like Software: A Promptfoo Deep Dive for QA
A production-level guide to testing LLM systems with Promptfoo: versioned evals, deterministic checks, judge-model bias, and provider benchmarking.
LLM-powered features break the core assumption of traditional QA: same input no longer guarantees same output, and multiple different responses can all be correct. This piece walks through using Promptfoo, the open-source eval and red-teaming framework, at a production-engineering level rather than as a one-off notebook script.
Key practices covered include pinning model versions instead of floating aliases to catch silent vendor-side regressions, exhausting deterministic assertions (JSON validity, regex, latency, cost) before reaching for model-graded checks, and mitigating judge-model bias — such as favoring longer or self-similar outputs — through cross-family judging and rubrics anchored to observable, falsifiable behavior rather than vague qualities like tone.
The article also introduces weighted scorecards for cross-provider benchmarking, since no single model wins across factuality, latency, cost, and instruction-following simultaneously. It closes by flagging that RAG pipelines fail in distinct ways a simple 'did it answer' test misses, each requiring its own dedicated assertion.