Deterministic AI: Not everything should burn tokens

This piece examines a common architectural mistake: routing deterministic, repeatable tasks through language models as if they were universal runtimes. A real-world example shows a team having an agent pull a metrics API daily, reformat JSON, and write results to a table — work a simple script would do identically every time, but instead billed per token and occasionally wrong because a probabilistic model was doing mechanical work.

The fix proposed is a clear sorting rule: judgment, interpretation, and ambiguity-handling belong with the agent; scheduled jobs, API calls, data transforms, and structured storage belong in the app layer. A particularly common misstep gets called out directly — teams using an agent's memory or notes as a substitute database for structured records like customer lists or inventory counts. This breaks down because notes have no schema, force full-context reads for single fields, and degrade as unstructured text grows. Notes remain the right tool, however, for genuinely unstructured context like preferences, tone, and learned patterns that shape judgment rather than get queried.

The article grounds this in Vybe's architecture, where agents build and operate actual applications rather than just chatting. Recurring work is scheduled at the app level, data pulls run as code behind endpoints, and state lives in a real database the agent queries deterministically. Case studies — Competitor Radar and Falcon, a competitive intelligence tool — show agents spending tokens only on the judgment call of whether a signal matters, while all monitoring, scheduling, and delivery run as cheap, reliable deterministic code, substantially cutting token costs.