kvcache-compression

vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm

Algorithm-System Co-design: accurate and efficient 2-bit KV cache quantization for LLM Inference.

RestoreKV: Recovering full-cache behavior under aggressive query-agnostic KV cache eviction.

KV Cache with PagedAttention vs PagedAttention + TurboQuant - experiments across token sizes comparing memory, latency, and accuracy.

KV-cache compression for LLMs: reference implementations of TurboAngle and TurboQuant codecs with Triton GPU kernels

Production-ready 2/4-bit KV Cache quantization for vLLM via Triton; 70% VRAM saving & 1.8x speedup

Umbrella for the decentralized cooperative AI training/inference architecture built on the prime-factored coordinate lattice and the dominance order. Theory + Systems + Roadmap papers, contracts, offload pattern.

Clean from-scratch math core for shannon-prime-lattice: KSTE encoder, Friedman sieve, ARM (HRR in CRT cyclotomic ring), CRT NTT primitives, Position-as-Arithmetic.

Prime Power Transformer: A Number-Theoretic Architecture for Compute

Clean from-scratch inference engine for shannon-prime-lattice. NTT-based attention, two-node CRT-sharded inference path, KSTE-encoded KV state.

LLM inference in C/C++ Including TurboQuant and Ternary models, referencing from PrismML-Eng & TheTom. Grab a Bonasi Ternary 8B model and test it out: https://huggingface.co/prism-ml/Ternary-Bonsai-8B-gguf