Why price per 1M tokens is a misleading AI metric
Comparing AI models by price per 1M tokens can mislead teams. Tokenizer differences and chain-of-thought efficiency matter far more than the sticker price per token.
The commonly cited '$X per 1M tokens' metric for comparing AI API costs is far less meaningful than it appears. Each frontier lab uses its own tokenizer, so identical text splits into different token counts across models—and even within the same lab. Anthropic's recent tokenizer change caused the same text to require 30% more tokens, quietly shifting effective pricing without an official price hike.
More critically, hidden 'chain of thought' reasoning tokens cause massive variance in token efficiency across models. Using data from the Artificial Analysis benchmark, the author shows that models with lower nominal token prices (like GLM-5.2) can end up costing more per completed task than seemingly pricier models (like GPT-5.5). DeepSeek V4 Pro stands out as a strong cost-efficiency outlier despite a lower intelligence score.
The takeaway for engineers is clear: measure actual cost per completed task rather than relying on per-token pricing when choosing models. Ignoring this distinction risks selecting inferior, less efficient models simply because their sticker price looks lower.