Skip to content

HBF Is For AI Model Weights, HBM Is For KV Cache

Buckle up as the ongoing memory- and HBM-related bottleneck is about to get materially decongested, thanks to SanDisk’s initiative to develop High-Bandwidth Flash (HBF) in concert with SK hynix, which should eventually lead to a bifurcation in tasks that are currently performed solely by the HBM. HBF can offer read bandwidth that is orders of magnitude higher than what a JEDEC HBM4 offers, but suffers on write endurance Before discussing the HBF standard, let’s first go over the memory wall issue that often comes up with HBM. HBM and the memory wall issue An AI model with 100 billion parameters […]