200K-Token LLM Serving on a 24 GiB Laptop

Open-weight models enhance local coding and reasoning, yet their context and execution state challenge laptop memory. JustFit introduces an MLX-based inference runtime that integrates KVExec for compressed KV execution, PhaseSwap for component residency, and StateTrans for state-preserving transitions. These mechanisms enable just-in-time materialization and release, significantly boosting local serving capacity. On a 24 GiB M4 Pro MacBook, tests show a remarkable increase in context handling, completing 196,608 input and 16,384 output tokens while achieving high performance in various workloads.