ModelMRI: A Local Debugger for Peering Inside LLMs, VLMs and Robot Policies

ModelMRI is an open-source, MIT-licensed tool that runs entirely on a local machine to expose the internal mechanics of LLMs, vision-language models and robot policies while they execute. Installed via pip, it serves a browser UI on localhost with no cloud dependency, telemetry or API key, and works with arbitrary nn.Module, TorchScript or GGUF checkpoints rather than a single hardcoded architecture.

The tool goes beyond simple attention visualization: it ranks attention heads by causally ablating them and measuring the resulting KL divergence, using three different baselines (zero, mean, resample) to make the choice of ablation method visible rather than hidden. It also implements activation patching across residual, attention and MLP pathways to locate where a factual answer is decided in the network, testing findings against random-baseline controls.

A cross-architecture comparison in the source material shows that where a fact 'lives' in a model is not a universal transformer property — three models showed an early-layer MLP / late-layer attention pattern, while a fourth (Qwen3-1.7B) inverted it. The tool reports measurement costs in forward passes and includes weight-scanning to check checkpoints for executable code before loading, alongside honest refusal messages when a measurement would be unreliable.