Back to feed
arXiv cs.LG
arXiv cs.LG
7/15/2026
How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit

How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit

Short summary

This paper audits six KV-cache compression methods under query-aware vs query-agnostic protocols, finding that query visibility dramatically changes method rankings. In the economically realistic query-agnostic setting (compress once, answer many), only KeyDiff consistently beats trivial baselines, and the widely deployed SnapKV loses to a simple keep-start-and-recent heuristic. The per-method performance drop correlates with how much query information each method's scoring signal uses.

  • •Matched-budget audit of six KV-cache compression methods across 144K+ evaluations
  • •Query-agnostic protocol (realistic reuse scenario) reshuffles rankings: SnapKV loses to trivial baseline
  • •Only KeyDiff consistently beats trivial baselines; performance drop tracks query visibility in scoring

Generated with AI, which can make mistakes.

#research-breakthrough#ai-tools
Read full article at arXiv cs.LG

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving
arXiv cs.CL
Ablation, Statistical Inference, and Validation for KV-Cache Compression
arXiv cs.LG
VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs
arXiv cs.CL
LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
arXiv cs.LG