» Tag
latency
13 postsLLM Latency Budgets: Speed Up AI Workflows Without Guesswork
A practical guide to building stage-level LLM latency budgets, workflow classes, and runtime enforcement for reliable AI products.
Prefill/Decode Disaggregation Can Worsen Tail Latency, Not Fix It
Splitting prefill and decode across GPU pools adds queues and KV transfer overhead that can worsen tail latency without careful control-loop design.
Measuring Linux Input Latency: X11 vs Wayland, VRR, DXVK
A custom click-to-photon device tests X11 vs Wayland, VRR, and DXVK low-latency on Linux, challenging widely repeated gaming latency myths.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comFrom 1 to 33 Anycast Sites at One Cloud Provider: A Staircase, Not a Curve
Measurement of 33 anycast sites within a cloud provider shows latency changes. Insights are valuable for engineers optimizing configurations.
Hiding Memory Latency in eBPF: Solutions to Avoid Stalling
Exploring methods to hide memory latency with eBPF and its importance for engineers.
Indexing the Data Lake for Online Point Queries
Spotify and others leverage RAP to optimize point queries in data lakes.
Understanding Latency Issues Through Data Visualization
Explore the role of data visualization in understanding latency issues. Learn to evaluate performance using CDF.
The Compounding Effects of Performance Improvements
Performance improvements are influenced by the accumulation of small inefficiencies. Each fix creates opportunities for further enhancements.
CTA-Pipelining: A Latency-Oriented Scaling Method for Multi-GPU Systems
CTA-pipelining enhances performance in multi-GPU systems by focusing on latency-oriented scaling.
Execution Latency in a Polymarket Trading Bot: Four Layers
A breakdown of the execution layer behind a Polymarket bot with 11,717 trades: signal timing, EIP-712 signing, CLOB submission, and why the T-90 cutoff exists.