PromptLedger v0.7 adds CI regression gates for prompt evaluation
PromptLedger v0.7 adds evaluation runs, metric comparisons, and policy-based regression gates for CI-safe prompt releases.
PromptLedger v0.7 extends prompt version control with a proper evaluation layer. Previously the tool could show what changed between prompt versions; now it can help answer whether a new prompt actually performed better. Benchmark results from external tools are stored as dedicated evaluation-run records attached to specific prompt versions, capturing metrics like accuracy, latency, and cost across multiple runs, models, and datasets.
New compare commands deterministically match compatible evaluation runs between two versions or labels such as prod and staging, showing metric deltas side by side. The headline feature is policy-based regression gates: JSON policies define acceptable thresholds for metric regressions, and the gate command returns a non-zero exit code when a threshold is breached — making prompt changes failable in CI pipelines just like a broken test.
The local dashboard now surfaces evaluation history alongside prompt diffs, and diffs themselves are more accurate thanks to a sequence-aware comparison algorithm. PromptLedger still doesn't execute prompts or call any model provider, keeping its scope local-first, deterministic, and provider-independent. Backed by 119 passing tests, v0.7 positions the tool as the missing release layer between prompt experimentation and production.