TurboQuant: Four Months After Google's 6x VRAM Claim
Google's TurboQuant algorithm aimed for a 6x memory reduction, but official code is still not released.
Four months ago, Google claimed to reduce AI memory usage by 6x. TurboQuant aims to compress the KV cache, but official code has yet to be released. In the meantime, the community has developed several forks that question TurboQuant's initial expectations, adopting a more cautious stance.