How Postgres-Backed Queues Scale to 30K Workflows per Second

Conventional wisdom holds that Postgres can't handle high-throughput queues and that teams must switch to dedicated systems like RabbitMQ or Redis. DBOS challenges that assumption, detailing how targeted database optimizations let Postgres-backed queues process 30,000 workflow executions per second across thousands of servers.

Three changes made this possible. First, using FOR UPDATE SKIP LOCKED eliminated worker contention when multiple processes try to dequeue the same rows, letting thousands of workers pull work concurrently without collisions. Second, making transaction isolation conditional—using REPEATABLE READ only when global flow-control limits are needed, and READ COMMITTED otherwise—removed serialization failures that previously throttled throughput above roughly 1,000 dequeues per second. Third, redesigning secondary indexes to be sorted and partial (maintained only while a workflow is enqueued, or only when a parent ID exists) cut CPU spent on sorting and autovacuum index maintenance.

For engineers building job queues or durable execution systems, this is a practical reminder that Postgres's built-in primitives—locking clauses, isolation levels, and selective indexing—can rival purpose-built queueing infrastructure at real production scale, without adding new operational dependencies.