In agent workloads with long contexts, such as Kimi K3's 1M token window, KV cache I/O scales linearly with context length, requiring the full historical KV to be read for every new token generated. This linear scaling of memory bandwidth demand quickly emerges as the primary bottleneck during the decode phase of long-horizon agent tasks.
Memory bandwidth, not fixed state manipulation, is the binding variable for long-context agent inference, driving demand for higher HBM capacity and bandwidth.