PXQ format revives Pascal/Volta GPUs: +88% prefill on a $150 Tesla P100

A developer forked ik_llama.cpp into pxq_llama, adding PXQ — a new quantization format plus a CUDA kernel set built specifically for pre-Turing datacenter GPUs like the Tesla P100, V100 and GTX 1080 Ti. In head-to-head benchmarks on a 35B, 256-expert MoE model (same card, same cold prompt, both engines at their documented best), pxq_llama beat upstream ik_llama.cpp by +88% prefill and +30% decode on the P100, +13% on both metrics on the V100, and +25% decode on the 1080 Ti — with one cell where upstream's more mature 2-bit prefill tile still wins, disclosed openly on the chart.

PXQ compresses MoE expert tensors using a learned codebook with fp16 anchors amortized over 64-row panels and a 4-bit sub-scale per 16-element block, across three tiers (PXQ2/3/4, 2.27-4.27 bits per weight). It pairs with fused CUDA kernels tuned for Pascal/Volta silicon — grouped-MoE GEMM, K-split decode GEMV, gate/up+GLU fusion, and a DeltaNet linear-attention decode fusion. A new int8 dp4a prefill tile for GTX 10-series cards pushed cold prompt processing on the 1080 Ti from 251 to 1,001 t/s, and a knapsack-based mixing mode lets llama-quantize pack PXQ2/3/4 per expert to exactly fit a given VRAM budget.

The author also documents a general finding: on pre-Turing cards, flash-attention off favors prefill while flash-attention on favors decode, for both engines — a regime split apparently undocumented elsewhere. As a control, running upstream's own IQ_K quants through the pxq_llama build with fusions only (no PXQ) still yielded +2.7-3.3% decode gains with bit-identical output to upstream, confirming the improvements come from the format and kernels rather than a weakened baseline. The full bench harness, raw CSVs, sha verification, and documented dead ends (e.g. a Pascal DMMV revival that cost -40%) are published under an MIT license.