Transformers Struggle with Arithmetic, So I Hand-Set Weights for Accuracy
An experiment shows a transformer can achieve 100% accuracy in arithmetic by setting weights manually without training.
I tested whether a transformer could perform exact arithmetic by manually setting its weights without training. The three-digit calculator accurately computes all 3,000,000 expressions, with checkpoints published on Hugging Face supporting up to 12-digit multiplication.