Implemented Qwen3.5's Mamba-Transformer Architecture from Scratch in Rust
Insights into the architecture and debugging of the ferrite inference engine for Qwen3.5, built from scratch in Rust.
Ferrite is a from-scratch inference engine developed for Qwen3.5-0.8B entirely in Rust. The architecture combines Gated DeltaNet and standard GQA transformer attention mechanisms. This write-up shares insights into the encountered challenges and debugging processes, highlighting key architectural decisions and their implications for engineers.