» Tag
ml-infrastructure
3 postsThe Real Cost of LLM Inference Is Memory Bandwidth
LLM serving costs are driven by memory bandwidth, not FLOPs. Why KV cache movement, prefill/decode disaggregation, and batching define real inference economics.
How Netflix Built Its Own In-House LLM Serving Stack
Netflix details its in-house LLM serving architecture: vLLM engine choice, Triton packaging, OpenAI-compatible API, and production rollout issues.
ZML/LLMD Alpha: One LLM Server Across CUDA, ROCm, TPU, Metal
ZML/LLMD alpha runs LLaMa, Gemma, Qwen and Mistral models across NVIDIA, AMD, TPU, Intel and Apple Metal in one server, with DFlash speeding up inference.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com