The bottleneck of agentic AI is no longer the GPU: it's the hard drive that feeds it

By SiliconANGLE (theCUBE) · July 8, 2026.

At the RAISE Summit 2026, Greg Matson, vice president of marketing and product at Solidigm —SK Hynix's brand for enterprise NAND—, gave an interview to theCUBE with a simple thesis: storage has stopped being cheap plumbing and become what he calls the 'intelligence layer.' According to Matson, a couple of years ago the focus was on training, which required high-capacity storage right next to the GPU. Now, with agentic inference, the pressure has soared: the new storage layer acts as an extension of the system's memory, storing and serving the context an agent accumulates as it reasons across multiple steps.

The figures he offers are illustrative, though they should be taken as the source presents them —an executive of Solidigm itself, in coverage that SiliconANGLE explicitly acknowledges is sponsored by the event—: a prompt of just 15 words can generate up to 40,000 output tokens, that is, between 5 and 10 gigabytes of context data. Multiplied by thousands of employees using agents at once, the storage requirement scales into the petabytes. Solidigm's commercial answer is SSDs of up to 122 terabytes per unit and what it says is the first cold-plate-cooled enterprise SSD, designed for fanless Nvidia GPU servers, at a time when AI racks are abandoning air cooling in favor of liquid.

It is worth separating the fact from the packaging. That an executive at a memory and storage manufacturer says storage is the most strategic piece of the AI stack is, by definition, a declared commercial interest; it is not an independent source. But the underlying phenomenon he describes is well documented in the sector through other channels: models with ever-longer context windows (SiliconANGLE itself reports, in the same batch of news, a 10-million-token model) and agentic workflows that chain dozens or hundreds of calls per task generate a volume of intermediate data —context caches, session states, tool histories— that no longer fits comfortably in GPU memory and needs a fast intermediate tier between the chip and traditional storage.

Our reading is that this news, though it springs from a product pitch, points to something we already sensed: the real cost of agentic AI is not generated by people writing prompts, but by agents orchestrating thousands of automatic calls per task, and it is that internal data traffic —not just the compute— that is rewriting the infrastructure investment map. During the training phase, the money and the narrative concentrated on the GPU and HBM; in the agentic inference phase, a growing share of spending shifts toward layers that used to be invisible: high-capacity storage, liquid cooling, internal networking. It is the same logic we already saw in the Google-Microsoft dispute over the 'plumbing' of agents: whoever controls the boring but indispensable layer of the system also captures value, even if it never appears in the headlines about the latest model.

In the short term, this translates into a capex cycle that gives no respite: hyperscalers have spent between 12 and 18 months replacing storage infrastructure more than a decade old, and that replacement is accelerating just as the deployment of sovereign AI by regional governments adds another simultaneous layer of demand. It is a real business for memory manufacturers like SK Hynix, but also a risk of overbuilding if enterprise agent adoption does not grow at the pace being financed; we have seen that pattern before with GPU capacity.

In the long term, however, this kind of infrastructure —however unglamorous it may be— is precisely what lowers the cost per token and makes it viable for agentic inference to stop being a laboratory luxury and become an accessible utility. Every improvement in how context is served without saturating the GPU is, ultimately, an improvement in how much intelligence can be delivered per dollar spent, and that is exactly the kind of efficiency that underpins the abundance thesis: not the promise of a smarter model, but the certainty that running already-capable models keeps getting cheaper for more and more people.