NexusQuant: E8 lattice KV cache compression

Calibration-free KV cache compression. Same model, same prompt, 3 cache modes. Watch quality hold as compression increases.

pip install nexusquant-kv | GitHub | Paper

FP16 is the baseline. E8 K3V2 is ~6x compression. E8 K2V2 is ~8x compression. Both are calibration-free.

Prompt

FP16 (baseline, 16 bpe)

E8 K3V2 (3-bit, ~6x)

E8 K2V2 (2-bit, ~8x)

Memory savings

Headline: +0.276% PPL at 5.83x compression (Mistral-7B, wikitext-2). Validated on 9 architectures. Source data