Mechanical Sympathy: Principles for Hardware-Aware Software Design
How CPU cache hierarchies, false sharing, and single-writer architectures underpin mechanical sympathy principles for building high-performance systems.
This piece revisits Martin Thompson's 2011 concept of 'mechanical sympathy'—borrowed from F1 champion Jackie Stewart—to explain why engineers who understand hardware behavior can build dramatically faster software. It walks through the CPU memory hierarchy (registers, L1/L2/L3 caches, RAM) and shows why sequential memory access vastly outperforms random access, with practical implications for things like ETL pipeline design.
The article also covers cache lines and false sharing, where multiple CPUs writing to different variables within the same cache line cause severe latency spikes, particularly affecting atomic variables; padding is presented as a common mitigation. The centerpiece is the single-writer principle: rather than protecting shared resources with mutexes, a dedicated actor thread should own all writes, communicating with other threads via asynchronous messaging. This is illustrated through refactoring a naive ONNX-based text embedding service into an actor-based architecture that batches inference calls, avoiding head-of-line blocking. The piece closes by introducing 'natural batching'—dynamically sizing batches based on queue state rather than fixed size or fixed interval—as a way to balance throughput and latency.
For engineers, the takeaway is concrete: understanding memory access patterns, cache-line behavior, and concurrency costs at the hardware level can matter more than algorithmic choices when building low-latency, high-throughput systems.