» Tag

gpu-inference

3 posts

Jul 12, 20268.43
Tesla P100's silent FP16 precision bug in llama.cpp fixed
A 3-line CUDA patch in llama.cpp fixes a years-old silent FP16 precision bug on Tesla P100 GPUs, with zero performance cost.
cuda gpu-inference llama.cpp local-llm
Jul 20, 20268.41
PXQ format revives Pascal/Volta GPUs: +88% prefill on a $150 Tesla P100
PXQ format and custom CUDA kernels in ik_llama.cpp fork pxq_llama deliver major prefill/decode speedups on Tesla P100, V100 and GTX 1080 Ti GPUs.
cuda gpu-inference llm moe
Jul 19, 20268.20
Custom Vulkan Engine Beats llama.cpp by 1.44x for One Model on RDNA3
A hand-written Vulkan inference engine for one model on RDNA3 GPUs decodes 1.44x faster than llama.cpp with token-exact output parity.
gpu-inference llm moe rdna3
Ad
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com