External key-value cache offloading can drastically reduce LLM inference costs and latency by avoiding redundant computations, making large-scale deploymen