Clustering Two Dell Pro Max GB10 Systems for Local AI Compute
Two Dell Pro Max GB10 systems clustered via RDMA/RoCE for 256GB of local AI memory, revealing setup quirks and firmware pitfalls.
Tom's Hardware linked two Dell Pro Max with GB10 systems using Nvidia's ConnectX-7 200Gbps networking to build an RDMA-over-Converged-Ethernet (RoCE) cluster, pooling 256GB of unified memory for running large language models that don't fit within a single 128GB system.
The motivation is cost and practicality: building an equivalent discrete-GPU server would require expensive Threadripper Pro or Epyc platforms, $20K+ in GPUs, and power/noise levels that are hard to live with. By contrast, the tested Pro Max GB10 configuration with a 4TB SSD costs roughly $6,332 per unit, and a pair delivers a large, network-scalable memory pool out of the box.
Getting full bandwidth wasn't trivial. The ConnectX-7 NIC only supports Ethernet/RoCE on these systems (not Infiniband), and its PCIe 5.0 x4 link has to be split into two logical interfaces per physical port to hit 200Gbps. A firmware update stuck behind a phasing gate initially throttled bandwidth well below expectations; forcing the update via apt and rebooting resolved it, bringing NCCL bandwidth tests close to the expected 25GB/s. The process worked, but it's far from plug-and-play.