Five ways your LLM cost tracking is lying to you
Five silent failure modes in LLM cost metering — streaming, prompt caching, serverless flush, cancelled streams, and stale pricing tables.
Your LLM provider's invoice tells you the total spend, not which feature or model drove it — so teams build a metering layer that wraps the client and multiplies usage by a price table. The trouble is this layer can silently produce wrong numbers with no exception ever thrown.
Five failure modes recur across implementations. Streaming responses from OpenAI return no usage data unless you explicitly request it, meaning every streamed call can register as free and even bypass budget gates. "Cached tokens" behave completely differently per provider — ignoring the cache breakdown causes over-counting on OpenAI and under-counting on Anthropic. Serverless runtimes like Cloudflare Workers or Lambda terminate background work the instant a handler returns, so fire-and-forget metering calls simply vanish.
On top of that, a stream cancelled mid-response still gets billed by the provider, but if recording logic sits after the consumption loop it never executes — real cost, zero record. And static price tables quietly return $0 for any unrecognized model, making cost dashboards falsely appear to drop the moment a team adopts a new model.
None of these failures throw errors — they just make dashboards and budget controls quietly wrong, which matters a great deal once financial or safety decisions depend on that data.