» Tag
speculative-decoding
4 postsLeakyLMs: Timing Attacks Expose LLM Architecture and Inference Tricks
LeakyLMs shows how timing side-channels can leak LLM architecture details and speculative decoding setups from production APIs like Gemini and Llama.
Bw24: From-Scratch Rust+CUDA LLM Engine Tuned for RTX 50 Blackwell
Bw24 is a from-scratch Rust+CUDA LLM inference engine tuned for RTX 50-series Blackwell GPUs, hitting up to 2.3x speculative decoding gains over llama.cpp.
Speculative Decoding: The Free Speedup Most Local LLM Setups Skip
Speculative decoding speeds up local LLM inference 1.5-2.5x with identical output; 2026 saw it built into models via multi-token prediction.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comDFlash in llama.cpp: 4.44x Faster Local Inference on Qwen 3.6 27B
DFlash, merged into llama.cpp, uses block-diffusion drafting to boost Qwen 3.6 27B inference speed, hitting 4.44x at 36K context with near-lossless quality.