LMCache Team's CacheBlend Research Improves KV Cache Reuse 2-4x for RAG

The CacheBlend research, awarded EuroSys 2025 Best Paper, addresses the inefficiency of traditional prompt caching where 90% of KV cache is unused due to rigid prefix matching rules in RAG and dynamic conversation histories. It reuses individual document caches and selectively recomputes only boundary tokens, leading to 2-4x faster processing on multi-document queries without quality loss. CacheBlend is implemented in LMCache, an open-source cache management layer integrating with vLLM, SGLang, and TensorRT-LLM on NVIDIA and AMD GPUs.