SwarmLLM: Peer-to-Peer LLM Inference Across Browser Tabs
SwarmLLM splits Qwen 3.8 27B across browser tabs via WebRTC and a custom WebGPU engine, matching native llama.cpp decode speed with zero install.
SwarmLLM splits large language models that no single device can hold across every machine in a room—laptops, phones, PCs—running each slice inside a plain browser tab over direct WebRTC connections. There's nothing to install and no server ever sees the conversation; only a 10 KB activation vector passes between peers per token.
The project ships a from-scratch WebGPU engine (~50 WGSL kernels) capable of running Qwen 3.8 27B (15 GB of Q4_0 weights) split across devices like a MacBook and an iPhone. On an NVIDIA GB10 it measures 9.0 tok/s in plain decode and 16 tok/s with speculative decoding, edging out native llama.cpp's 8.0 tok/s on the same file and GPU—while golden tests keep every optimization bit-exact with plain decoding.
What matters for engineers: unlike exo, Petals, distributed-llama or llama.cpp's rpc-server, SwarmLLM requires no native binaries, Python packages, or open ports—just a WebGPU-capable browser tab. Its memory-bandwidth-bound decode design and verified speculative decoding path suggest browser-based inference can now be competitive with native runtimes, not just a convenience layer.
This synthesis was produced from its source by AI; there is no human editor or manual review step. How we work