Token Drift Explained: Why AI Agents Get Slower and Pricier
A breakdown of token drift in LLM agents: why session costs grow quadratically and how context budgets keep long-running agents efficient.
An LLM agent that feels snappy in a demo can slow down and get noticeably more expensive by turn twenty of a real session. The cause isn't the model degrading — it's that the effective input context sent on every call keeps growing as conversation history, tool outputs, retrieved documents, and state accumulate. This effect, called token drift, is quantified here with a simple cost model showing that cumulative session cost can scale quadratically with turn count, even when each individual turn only grows linearly.
The growth isn't limited to chat history: large tool schemas, oversized tool results, stale or excessive RAG retrievals, leftover retry/error traces, and duplicated facts across summaries and raw transcripts can all inflate context independently. This is why capping the number of messages is an incomplete fix — a short conversation can still be expensive if one tool call returns a huge JSON payload.
The piece lays out concrete engineering practices: measure real token usage via provider APIs rather than character estimates, enforce a token budget (not just a message-count limit) that reserves space for output and tool calls, treat a full agent turn as the atomic unit when trimming context, and separate durable structured state from prose summaries and raw history. Together these practices give teams a way to keep long-running agents both affordable and reliable as sessions scale.