Regex Beats LLM Classifiers by 45 Points in Agent Context Routing
In a new test, plain regex rules beat TF-IDF classifiers by 45 points at routing agent context, showing an LLM call is often unnecessary.
LLM coding agents waste enormous amounts of tokens re-opening the same files and wandering through repos without memory. A July 2026 paper, ContextSniper, tackles this in repository-level program repair with tiered memory plus an intention-aware 'context gate' that filters low-value regions before they reach the model, cutting token usage by 51.5% on one agent and 38.9% on Claude Code while keeping resolution rates roughly flat.
That gate is essentially a classifier deciding which retrieval strategy (symbol lookup, semantic search, graph impact, etc.) fits a request. To test whether this decision actually needs an LLM, the author built 140 hand-labeled requests across seven intent classes and ran five-fold cross-validation comparing three cheap approaches. A roughly 40-line regex heuristic hit 94.3% accuracy and 0.945 macro-F1, beating TF-IDF centroid and logistic regression models — both around 48% — by roughly 45 points.
The explanation is that intent here lives in the shape of a request, not word frequency: camelCase tokens signal symbol lookup, 'how does X work' signals semantic search, 'rename' or 'set to' signals mutation — patterns regex captures directly while sparse TF-IDF models lose on a tiny dataset. Only about 3% of requests were genuinely ambiguous, and even those were better resolved by probing the cheapest grounded lookup first and falling back to semantic search, rather than escalating to a bigger model.
The practical takeaway for engineers building agent context systems: put regex first, add probe-and-fallback for ambiguous cases, and reserve LLM calls as a last resort for the small residue that survives both. The author flags important caveats — the rules and test corpus share an author, so the 94.3% figure is in-distribution and needs validation on real production traces — and notes ContextSniper's own official numbers (24% vs. 26% resolved) as convergent but not identical evidence that context gates matter.