1source_id,title,authors,venue_status,year,url,access_date,revision,repository_url,commit,checkpoint_availability,supported_backends,license,relevance,reproduction_status,verification2kivi2024,KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache,"Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, Xia Hu",ICML 2024 (stated in arXiv Comments and repo),2024,https://arxiv.org/abs/2402.02750,2026-10-29,v2 (2024-07-25),https://github.com/jy-yuan/KIVI,876b4d2d08e3b1d5f70d0969c299d8c7c42ddfb6 (2025-11-20),"none required (training-free; applies to HF Llama-2/3, Mistral, LongChat checkpoints)","PyTorch + custom CUDA extension (kivi_gemv) + Triton packing kernel + flash-attn; requirements.txt pins torch==2.1.2, transformers==4.36.2, flash-attn==2.5.6, triton==2.1.0 (README: compatible with transformers 4.43 since 2025-01); CUDA only; no Apple Silicon support",MIT,"asymmetric 2-bit KV quantization (keys per-channel grouped over G=32 tokens, values per-token, fp16 residual window R=128); question-agnostic lossy compression baseline; orthogonal to eviction",pending,"arXiv abstract + HTML v2, GitHub page, raw README/llama_kivi.py/requirements.txt/quant/setup.py/example.py, GitHub API fetched 2026-10-29"3snapkv2024,SnapKV: LLM Knows What You are Looking for Before Generation,"Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, Deming Chen","NeurIPS 2024 main track per proceedings.neurips.cc (secondary); NOT stated on arXiv page (no Comments) nor in repo (cites arXiv preprint)",2024,https://arxiv.org/abs/2404.14469,2026-10-29,v2 (2024-06-17),https://github.com/FasterDecoding/SnapKV,e216ddc84c5bd210378cbdbbba12ba02102aa640 (2025-07-10),"none required (training-free monkey-patch; configs for Llama-2-chat, LongChat-7b-v1.5-32k, Mistral-7B-Instruct, Mixtral-8x7B-Instruct, LWM-Text-Chat-1M)","PyTorch + HF transformers monkey patches (Llama, Mistral, Mixtral); README tested transformers==4.37.0, requires transformers>=4.36 and flash-attn==2.4.0 (pyproject: transformers>=4.34, torch unpinned); CUDA via flash-attn; no Apple Silicon mention; core selection is plain PyTorch",Apache-2.0,"question-aware eviction: last-window (16/32/64 tokens) attention votes + 1D pooling (kernel 5/7/13) + per-head top-k over prefix; the observation window is the end of the prompt (instruction/question); primary question-aware baseline",pending,"arXiv abstract + HTML v2, GitHub page, raw snapkv_utils.py/pyproject.toml/pred_snap.py/LongBench configs, GitHub API, NeurIPS proceedings page fetched 2026-10-29; repo not linked from arXiv page (found at expected URL, BibTeX matches)"4pyramidkv2024,PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling,"Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, Wen Xiao","COLM 2025 per Microsoft Research page (secondary); NOT stated on arXiv page (no Comments) nor in repos (BibTeX: CoRR 2024 / arXiv preprint)",2024,https://arxiv.org/abs/2406.02069,2026-10-29,v4 (2025-05-15),"https://github.com/Zefan-Cai/KVCache-Factory (canonical, renamed from Zefan-Cai/PyramidKV on 2024-11-28); https://github.com/BartsLab/PyramidKV exists but is a 0-star fork snapshot","KVCache-Factory 68cd9551a63fedf362ddb0edb008badd04c24996 (2026-08-12); BartsLab fork 506c866a876bdcc7ca4f2f73aa7f856a8c666479 (2024-11-05)","none required (training-free; Llama-3-8B/70B-Instruct, Mistral-7B-Instruct-v0.2)","PyTorch + HF transformers; requirements.txt pins transformers==4.44.2 (README says >=4.41); flash-attn>=2.4.0.post1 optional (sdpa/eager supported); Llama + Mistral; batch size 1; no decoding-stage compression; no Apple Silicon mention; selection is plain PyTorch",MIT,"layer-wise budget pyramid (arithmetic from 2*base-base/beta at layer 0 down to base/beta at top layer, beta=20) on top of SnapKV-style question-aware selection (window alpha=8, kernel 7 maxpool in runner); question-aware; budget rule itself is content-independent",pending,"arXiv abstract + HTML v4, GitHub pages (BartsLab fork + KVCache-Factory), raw README/LICENSE/requirements.txt/run_longbench.py/pyramidkv_utils.py, GitHub API, Microsoft Research page fetched 2026-10-29; author list differs between arXiv v4 (Yucheng Li) and repo BibTeX/Microsoft page (Baobao Chang)"5h2o2023,H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models,"Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen",NeurIPS 2023 (stated in repo description '[NeurIPS'23]'; arXiv page has no Comments field),2023,https://arxiv.org/abs/2306.14048,2026-10-29,v3 (2023-12-18),https://github.com/FMInference/H2O,ac75c2a8a9e76832b2a4139b9363373b56336bfb (2024-06-18),"none required (training-free eviction; OPT via FlexGen, LLaMA via HF; paper also GPT-NeoX)","PyTorch >=1.12; h2o_flexgen (FlexGen fork, OPT) and h2o_hf (HF transformers installed unpinned from git main; lm-eval; crfm-helm); simulation (attention masking) and real KV drop (utils_real_drop); batch size 1; CUDA assumed; no Apple Silicon mention; eviction logic is plain PyTorch","MIT badge in README only; NO LICENSE file in repository (GitHub API license=null) — unverified as formal license","cumulative-attention eviction: per-head/per-layer accumulated softmax attention received + always-keep recent window; budget 20% of prompt split evenly heavy/recent (0.1/0.1); query-agnostic policy, question-informed only if the question is in the prefill; intermediate baseline between StreamingLLM and SnapKV",pending,"arXiv abstract + HTML v3 (2 of 3 chunks), GitHub page, raw README/h2o_hf README/h2o_flexgen README/utils_real_drop/modify_llama.py/utils_hh/modify_llama.py, GitHub API + contents listing fetched 2026-10-29"6streamingllm2023,Efficient Streaming Language Models with Attention Sinks,"Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis",ICLR 2024 (stated in arXiv Comments and repo description),2023,https://arxiv.org/abs/2309.17453,2026-10-29,v4 (2024-04-07),https://github.com/mit-han-lab/streaming-llm,2e5042606d69933d88fbf909bd77907456b9b4dd (2024-07-11),"none required (training-free; Llama-2, MPT, Falcon, Pythia/GPT-NeoX; demo lmsys/vicuna-13b-v1.3); sink-token pretrained 160M models not released on page","PyTorch (unpinned), Python 3.8, transformers==4.33.0 pinned; pos-shift patches for llama/gpt_neox/falcon, MPT via ALiBi; CUDA demo; no Apple Silicon mention (third-party iPhone integration cited in News); cache logic is pure tensor slicing",MIT,"position-only eviction: keep start_size=4 attention sinks + rolling recent window (recent_size 512 default / 2000 demo / 2044 paper); keys cached un-rotated and re-rotated with positions 0..kv_len-1 within the cache; canonical question-blind baseline",pending,"arXiv abstract + HTML v4, GitHub page, raw README/kv_cache.py/pos_shift/modify_llama.py/enable_streaming_llm.py/run_streaming_llama.py/setup.py, GitHub API fetched 2026-10-29"7