Wattage: An Offline Token-Cost Profiler and CI Gate for AI Agents

Wattage is an open-source CLI that ingests OpenTelemetry GenAI traces and turns raw token counts into priced, dollar-denominated findings. It runs eight detectors — covering prefix churn, cache gaps, verbosity, redundant tool calls, non-convergent loops, retrieval thrash, model mismatch, and reasoning overspend — against a vendored pricing snapshot, entirely offline with no API key required.

Its standout piece is a convergence engine that flags agents stuck in unproductive loops, including cases a naive exact-match duplicate detector cannot catch, such as retries with fresh timestamps or oscillation between two strategies. On a hand-labeled benchmark of 10 synthetic loops, Wattage's classifier hit perfect precision and recall, versus just 0.14 recall for a SHA-256 exact-match baseline.

Beyond the terminal report and shareable HTML flame graphs, Wattage ships a GitHub Action that fails CI when a pull request pushes token efficiency below a set threshold, cost delta above a percentage, or introduces any critical finding — posting a per-detector delta as a PR comment and emitting SARIF and JUnit XML. A separate post-merge workflow refreshes the committed baseline, keeping the regression gate honest over time.

For teams shipping LLM agents, this turns token spend from an opaque dashboard nobody checks into a debuggable, testable engineering signal backed by an actual CI gate.