Hidden Bugs Killing Claude Prompt Caching: The Fix That Cut Costs 80%

A developer discovered that Claude API prompt caching, meant to slash costs, wasn't working at all — cache_read_input_tokens stayed at zero across identical requests. The root cause was that caching relies on exact byte-for-byte prefix matching, and since content renders in a fixed order (tools, then system, then messages), any change at the start invalidates everything downstream. Three silent bugs were responsible: a timestamp embedded in the system prompt, JSON.stringify calls without sorted keys, and a tool list built dynamically per user.

Each of these caused the prefix to produce different bytes on every request, completely disabling caching. The fixes were straightforward: move the timestamp into the user message, sort JSON keys before serialization, and use a single stable, deterministic tool list for all users instead of a per-user one. Adding one cache_control breakpoint on the final system block then let tools and system prompt cache together.

Economically, cache reads cost roughly a tenth of base input pricing while writes cost about 1.25x, so savings kick in from the second request onward. In this case — where the system prompt and instructions stayed identical across a session — input costs dropped by around 80%. The takeaway for engineers building on LLM APIs: verify actual cache behavior via usage metrics, and audit prompt-construction code for anything non-deterministic sitting in the prefix.