The AI industry's shift from training to inference is driving a structural change in memory architecture, with vendors exploring High Bandwidth Flash (HBF), SSD POD, and SRAM to overcome HBM's capacity and bandwidth limitations for KV Cache. SanDisk and SK Hynix partnered in August 2025 to develop HBF, projected to offer 512 GB capacity at 1.6 TB/s bandwidth, while NVIDIA introduced SSD POD in 2026 and Google unveiled the inference-specific TPU v8i.
AI inference memory is diversifying beyond HBM, creating new demand for specialized flash and SRAM solutions to address KV Cache capacity and bandwidth bottlenecks.