» Tag
moe
16 posts397B MoE Model Runs on a Single RTX PRO 6000 96GB GPU
Krasis runtime runs a 397B-parameter MoE model on one RTX PRO 6000 96GB GPU, hitting 2,354 tok/s prefill via CPU-VRAM expert streaming.
Transformers vLLM Backend Now Matches Native Inference Speed
The transformers vLLM backend now matches native inference speed via torch.fx graph analysis and automatic kernel fusion for LLMs.
Thinking Machines Lab Ships Inkling: 975B-Param Open MoE, Tunable Reasoning
Thinking Machines Lab's 975B-param, 41B-active Inkling model offers encoder-free multimodality and a tunable reasoning-effort control.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comColibri lets 744B-parameter GLM-5.2 run on just 25GB of RAM
Colibri is a single-file C engine that runs GLM-5.2's 744B MoE model on 25GB RAM with no GPU, streaming experts from NVMe on demand.
GLM 5.2 and the Open-Source Shakeup of AI Profit Margins
Zhipu AI's open-source GLM 5.2 rivals GPT-4o performance via MoE architecture while slashing inference costs, reshaping AI's profit margin economics.
Running Kimi K3, a 2.8T-Parameter MoE Model, on an M1 Mac
Deltafin runs Kimi K3, a 2.8T-parameter MoE model, on a 64GB M1 Mac via full local install or expert streaming, no cluster required.
MiniMax M2.7: Open-Source AI That Optimized Its Own Training
MiniMax's open-source M2.7 model actively shaped its own training via memory writes and skill-building, gaining 30% efficiency and matching GPT-5.3-Codex on SWE-bench Pro.
PXQ format revives Pascal/Volta GPUs: +88% prefill on a $150 Tesla P100
PXQ format and custom CUDA kernels in ik_llama.cpp fork pxq_llama deliver major prefill/decode speedups on Tesla P100, V100 and GTX 1080 Ti GPUs.
MoE Expert Streaming Runs a 120B-Parameter LLM on an Android Phone
An open-source Android app streams MoE experts from flash storage to run a 120B-parameter LLM on phone CPUs, with 30B models at usable speeds.
Local Model Showdown Round 9: Qwen 3.6, Nemotron, Qwythos Coding Test
Five LLMs face off on a real coding task via llama.cpp on an RTX 5090 in Round 9 of the Local Model Showdown series.