Persistent State Machines Recast LLM Attention as Hardware FSMs

A new research paper reformulates the attention operator in large language models using a 'Persistent State Machine' (PSM) framework, where computation is broadcast as instructions to stationary in-memory cells running local deterministic state transitions. The work provides complete mathematical proofs covering quantization error bounds, a multi-phase discrete Softmax construction under a bounded-logits assumption, deterministic finite-automaton (DFA) equivalence, and spatial factorization into O(N) circuit size.

A two-phase engine—local score evaluation plus binary-tree reduction with global max extraction and local exponential lookup—was implemented in synthesizable RTL and run through the AMD Vivado 2026.1 flow (synthesis, placement, routing, static timing analysis, and post-implementation power estimation) targeting a Zynq-7000 xc7z020 device. The tool-estimated normalized dynamic energy came to 0.0267 pJ/op at an Fmax of 283.8 MHz, with functional simulation across over a thousand random test vectors showing bit-exact agreement against a fixed-point Python reference.

For hardware and efficiency-focused engineers, the key caveat is that these figures are Vivado tool estimates for synthesized logic only—no physical FPGA board execution or silicon measurement was performed, and system-level memory energy is excluded. Still, the approach points toward a path for shifting LLM inference from token-based software computation to dedicated hardware built on deterministic state machines.