LLM agents are breaking 20-year-old web architecture

For two decades, web services have been built on one assumption: state lives in the database, while application servers stay stateless. Scaling meant growing or sharding the database and horizontally replicating servers — any request could hit any server because the database was the single source of truth. LLM-powered agents quietly break that assumption in three ways: long-running work that behaves like an async process rather than a single request, stateful compute where an agent accumulates context across turns and tool calls (its memory, not database state), and bi-directional interaction where users want to watch, interrupt, and steer a running process rather than just query an API.

Durable execution frameworks like Temporal and Inngest make that long-running work resilient, but the underlying architecture still assumes statelessness — and that doesn't solve routing. HTTP, load balancers, and stateless servers can only route to a database, not to a specific running process. That forces everyone back to polling: repeatedly querying a database for updates a durable workflow wrote, at the cost of latency, wasted requests, database load, and poor UX for anything resembling a live stream.

The proposed fix is a routing primitive independent of any server: pub/sub channels. Unlike WebSockets, which are a connection (and lose their 'address' the moment they drop), a pub/sub channel is itself the durable address — both client and workflow process connect to it by name and can reconnect after a disconnect without losing continuity. The author argues this isn't really an LLM-specific issue; LLMs just make the gap painfully visible because their responses are non-deterministic and costly, unlike cheap, retryable, deterministic requests of the past. The takeaway for engineers: durable execution, pub/sub, and stateless HTTP each need to handle their own concern in a new architecture built for agentic, stateful workloads.