Why the AI Bill Grows Inside the Agent Loop
Token-saving tricks barely move the needle on AI agent costs; the real spend comes from retries, agent turns, and parallel subagents. A look at Copilot's billing shift.
The piece argues that the current obsession with context engineering and token-saving tricks—shorter prompts, /compact, disabling MCP tools—is looking in the wrong place. The real cost driver, per the author's "agent loop economics" model, lies in the larger multipliers: how many tasks get routed to AI, how many retries and failed attempts occur, how many turns an agent takes, and how many subagents run in parallel. Trimming a few thousand tokens from a prompt can feel responsible while doing almost nothing once attempts, agent turns, or parallelism start compounding.
A second issue is that the billing meter itself keeps shifting. GitHub's move from premium-request units to token-based AI Credits for Copilot is cited as a clear case: a quick chat question and a multi-hour autonomous coding session can no longer cost the same. Changes like Anthropic's newer tokenizer producing roughly 30% more tokens for identical text show that bills can grow even when user behavior doesn't change. The practical takeaway is to treat vendor pricing and measurement as versioned dependencies, re-baselining workflows whenever the meter changes.
Finally, the article distinguishes who actually holds the cost levers. End users of packaged agent tools like Claude Code, Copilot, or Cursor can scope tasks, cap loops, and pick models, but can't touch provider-level levers like prompt-cache tuning or hidden system scaffolding. Those building API products or gateways, by contrast, control prompt layout, caching strategy, and per-key budgets directly. For engineers, the message is to track task volume, retry rates, and subagent parallelism instead of chasing token-level savings, since that's where the real spend actually lives.