Frontiers of compute: how to cut AI inference costs

By McKinsey & Company.

This email, sent by Bill Wiseman and Marc de Jong, global leaders of McKinsey's Semiconductors practice, announces a new article titled "Frontiers of compute: The technologies to reduce AI inference costs".

The central message the email previews is brief but forceful: AI's next big breakthrough may not be a smarter model, but a cheaper token. In other words, the article focuses on the technologies that make inference cheaper (the process of running an already trained model to generate responses), as opposed to the usual emphasis on training ever larger models.

The body of the email does not develop any additional content on which specific technologies the full article addresses; it merely presents the headline, the hook line, and links to the main piece along with two related articles suggested under "Also Consider": "The next era of semiconductor value creation" and "Where AI will create value—and where it won't".

Broadly speaking, as sector context, reducing inference costs is a topic of growing strategic relevance for the semiconductor and agentic AI industry, since the mass deployment of agents and LLM-based applications depends largely on the cost per inference token continuing to fall, which directly affects the economic viability of scaling these systems in production.