NVIDIA ModelExpress: P2P RDMA Cuts Model Startup From Minutes to Seconds

Large language model checkpoints now routinely span hundreds of gigabytes to a full terabyte, and modern serving pipelines move these weights constantly during cold starts, autoscaling, rolling deployments, and continuous RL post-training weight syncs. Each of these workflows pays the same tax: time spent transferring bytes before a GPU can do useful work.

NVIDIA ModelExpress (MX) addresses this by first checking whether a compatible copy of the weights already exists on another serving replica. When a peer is found, MX transfers tensors directly from GPU to GPU over P2P RDMA using the NVIDIA Inference Xfer Library (NIXL), skipping object storage, disk, and host memory entirely. When no peer exists, it bootstraps via the fastest available path, streaming safetensors straight from object storage into GPU memory or reading through GPUDirect Storage, without ever landing the checkpoint on local disk.

In practice, MX transferred DeepSeek-V4 Pro weights and JIT kernel caches between replicas in under 10 seconds, cutting total startup time from roughly 8 minutes to 1 minute 44 seconds. Additional optimizations, including deduplicating concurrent cluster-wide downloads through a shared cache service and cutting NIXL memory-registration overhead by up to 99 percent, further shrink the cost of scaling out large models.

For engineers running large-scale inference or RL training pipelines, MX turns weight distribution from a repeated cold-start penalty into a one-time bootstrap followed by fast GPU-to-GPU fan-out, directly reducing autoscaling latency and rollout synchronization overhead.