HBF: high-bandwidth flash memory to accelerate agentic LLM serving (UC Berkeley, FuriosaAI)

A team of researchers from UC Berkeley and the AI chip startup FuriosaAI has published a technical paper titled "Characterizing High Bandwidth Flash for LLM Serving," focused on an increasingly pressing problem in artificial intelligence infrastructure: memory. The work, authored by Zack Yu, Chloe Wong, Coleman Hooper, Minjae Lee, Wonjun Kang, Youngjin Cho, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer and Amir Gholami, was published as a preprint on arXiv in September 2026.

The paper's starting point is a bottleneck well known in the industry: serving large language models (LLMs) requires storing both the model weights and the so-called KV cache (the cache of keys and values that the model generates and reuses during text generation). As models grow and context windows get longer, both memory capacity and bandwidth become performance limiters. This problem is especially acute with "agentic" workloads —AI systems that sustain repeated interactions and accumulate context progressively—, where preserving the KV cache state for reuse becomes increasingly important.

The approach analyzed in the study is the use of HBF (High-Bandwidth Flash), a technology that seeks to expand the memory capacity of AI accelerators beyond what conventional HBM (High-Bandwidth Memory) allows today. According to the source, high-bandwidth flash carries two practical problems that complicate its use: its access costs and a limited write lifetime (flash devices degrade with write cycles), which makes direct integration into model-serving systems difficult.

To assess when and how this additional capacity actually improves performance and energy efficiency, the authors designed two key pieces. The first is a three-tier hierarchical storage system —HBM, HBF and host memory— that coordinates data placement across those tiers. The second is a "buffered cache-aware scheduling" technique, aimed at taking advantage of HBF's capacity benefits without prematurely exhausting its write lifetime. All of it was evaluated through trace-driven simulations of the workloads considered in the study.

The results reported in the abstract are significant. According to the study, faster HBF-augmented systems reduce task completion time by between 36.1% and 87.0% compared with systems relying solely on HBM, depending on the workload evaluated. On energy efficiency, modeled energy savings reach up to 55.8%, although the authors themselves caveat that in some "light" workloads the use of HBF may actually increase energy consumption, suggesting that this technology is not a universal solution but rather depends on the workload profile. Finally, and in relation to the write-degradation problem, the proposed scheduling technique extends HBF's estimated write lifetime from 4.77 to 14.82 years in the evaluated configuration, a substantial improvement that suggests careful management of where and when writes occur can be as important as the memory technology itself.

Taken together, the work suggests that high-bandwidth flash memory should not be thought of as a simple replacement for or passive complement to HBM, but as one more tier within a memory hierarchy that requires active coordination between data placement and scheduling decisions to be viable in the long run. For an industry that has spent months debating how to scale the memory capacity of AI accelerators without driving up data center costs and energy consumption, this research provides quantitative evidence —albeit based on simulation, not real large-scale deployments— that hybrid HBM-HBF-host solutions, properly orchestrated by software, could ease some of the pressure created by ever-longer contexts and the agentic workflows that dominate the generative AI roadmap.