Rust WebRTC SFU moves to thread-per-core: 70ms to 10ms latency

PulseBeam, an open-source Rust WebRTC SFU, migrated from Tokio's default multi-threaded work-stealing scheduler to a thread-per-core (TPC) architecture, dropping P99.99 end-to-end latency from a spiky 70ms to a stable 10ms while raising system capacity by 25%. The root cause was excessive async task spawning per packet: each hop between demux, per-participant processing, and fan-out was a separate task connected via channels, creating yield points where the scheduler could preempt a nearly-finished packet in favor of a newer one, effectively causing priority inversion.

Because the team already used the sans-I/O str0m library, they could swap concurrency models without rewriting the media stack. By removing channel hops entirely, each connection's full pipeline—from socket ingress through processing to egress—now runs synchronously on a single task pinned to one CPU core. This trades some concurrency for predictable execution, eliminating cross-core contention and improving cache locality, echoing the shard-per-core designs used by systems like Redpanda and ScyllaDB. This differs from LiveKit's Go-based work-stealing approach and mediasoup's process-per-core isolation model, with PulseBeam opting for a lean, lock-free design without multi-process overhead.

The new setup uses a Tokio LocalRuntime per thread, plus a dedicated control thread for SDP negotiation and traffic admission, with inter-shard communication limited to at most one channel hop. Benchmarks on four isolated, underclocked i9-12900HK P-cores—scaling up to 120 rooms of 4 participants each (480 concurrent users)—showed marked improvements in both ICE round-trip time and end-to-end latency. The results offer a concrete case study in how task proliferation and channel-hop scheduling can dominate tail latency in real-time media servers.