Pith Number

pith:6S2KKZLV

pith:2024:6S2KKZLV64O6J4OIQJLMDQIB4U

not attested not anchored not stored refs resolved

Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference

Junlin Lv, S. Kevin Zhou, Xike Xie, Yuan Feng, Yukun Cao

A theoretical upper bound on attention loss from KV cache eviction enables adaptive per-head budget allocation.

arxiv:2407.11550 v5 · 2024-07-16 · cs.CL · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{6S2KKZLV64O6J4OIQJLMDQIB4U}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

we establish a theoretical loss upper bound between pre- and post-eviction attention output, explaining the optimization target of prior cache eviction methods, while guiding the optimization of adaptive budget allocation. Base on this, we propose Ada-KV, the first head-wise adaptive budget allocation strategy.

C2weakest assumption

The derived loss upper bound accurately captures the quality impact of eviction and that attention heads exhibit sufficiently distinct patterns to benefit from non-uniform budget allocation without introducing new approximation errors that undermine the bound.

C3one line summary

Ada-KV is the first head-wise adaptive KV cache budget allocator for LLMs, using a theoretical loss upper bound to allocate eviction differently per attention head and yielding higher quality than uniform methods on long-context benchmarks.

References

69 extracted · 69 resolved · 16 Pith anchors

[1] A survey on recent advances in llm-based multi-turn dialogue systems 2024

[2] Summedits: measuring llm ability at factual reasoning through the lens of summarization 2023

[3] Llm-based code generation method for golang compiler testing 2023

[4] GPT-4 Technical Report 2023 · arXiv:2303.08774

[5] The claude 3 model family: Opus, sonnet, haiku, March 2024 2024

Formal links

2 machine-checked theorem links

Cited by

40 papers in Pith

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling

RoPE-Aware Bit Allocation for KV-Cache Quantization

Recency/Frequency Adaptive KV Caching for Large Language Model Serving

QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

Receipt and verification

First computed	2026-05-17T23:38:14.260301Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

f4b4a56575f71de4f1c88256c1c101e52500e026ad0ac91a66bfe3487091362d

Aliases

arxiv: 2407.11550 · arxiv_version: 2407.11550v5 · doi: 10.48550/arxiv.2407.11550 · pith_short_12: 6S2KKZLV64O6 · pith_short_16: 6S2KKZLV64O6J4OI · pith_short_8: 6S2KKZLV

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/6S2KKZLV64O6J4OIQJLMDQIB4U \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: f4b4a56575f71de4f1c88256c1c101e52500e026ad0ac91a66bfe3487091362d

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "d97adabc45e8212c7ade479ef329942d4caf3844171883c7112194e1e9466758",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2024-07-16T09:53:32Z",
    "title_canon_sha256": "467a4db061812a003b56fa4551f1967a4777d9f127f2d8e30f7c49e751d72a89"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2407.11550",
    "kind": "arxiv",
    "version": 5
  }
}