Running Kimi K3, a 2.8T-Parameter MoE Model, on an M1 Mac

Deltafin is a research project that runs Kimi K3, a 2.8-trillion-parameter Mixture-of-Experts model, on a single Apple Silicon Mac with 64 GB of RAM. The model's full weights total roughly 1.56 TB, far exceeding both the machine's RAM and typical disk budgets, but MoE sparsity means each token only touches a fraction of the network: 16 of 896 experts per layer across 92 layers.

The project offers two installation modes. A full install downloads all ~1.7 TB of expert data locally, taking 5-10 hours but yielding steady 60-76 seconds per token with no network dependency at inference. A streaming mode needs only ~215 GB and about 30 minutes to set up, but fetches uncached experts via HTTP range requests during inference, pushing latency to 3+ minutes per token for anything not already cached. Users can start with streaming and upgrade to a full local install later without reconfiguration.

Deltafin reuses Moonshot's unmodified modeling code through a pure-PyTorch shim that replaces CUDA-only kernels, adds a fused MXFP4 GEMV kernel compiled with clang, and exposes an OpenAI-compatible API server. Configuration is largely automatic (GPU detection, int8 spine quantization, speculative decoding), though the project is explicit that this is slow, greedy-only inference meant for experimentation rather than production workflows.