Moonshot AI Ships Kimi K3: A 2.8T-Parameter Open MoE Model

Moonshot AI has released Kimi K3, described as the world's first open 3-trillion-parameter-class model. The system carries 2.8 trillion total parameters, native vision support, and a 1-million-token context window.

Three architectural components stand out. Kimi Delta Attention, a hybrid linear attention mechanism, breaks conventional prefix caching and has been upstreamed into vLLM, delivering up to 6.3x faster decoding at million-token context lengths. Attention Residuals operates along the depth axis rather than sequence length, selectively retrieving representations across layers to boost training efficiency by roughly 25% at under 2% added compute cost. Stable LatentMoE, running at 16-of-896 sparsity, replaces heuristic expert-routing updates with Quantile Balancing, deriving allocation directly from router-score quantiles and yielding about 2.5x scaling efficiency over the previous K2 model.

On published benchmarks under maximum reasoning effort, Kimi K3 scored 91.2 on BrowseComp, 88.3 on Terminal Bench 2.1, and 77.8 on Program Bench, outperforming Fable 5 and GPT 5.6 Sol on 6 of 35 reported rows. It trails Fable 5 on FrontierSWE (81.2 vs 86.6) and HLE-Full (43.5 vs 53.3). For engineers, the release matters for both its long-context inference performance and for open-sourcing novel solutions to MoE routing stability and attention scaling at trillion-parameter scale.