The future of AI may be small, cheap and unprofitable: Joachim Klement

The AI boom is built on the idea that bigger is better. A recent study suggests the opposite may soon be true: small language models running on desktop computers may be able to handle most of the tasks currently performed by large language models.

If that's true, Anthropic, OpenAI and SpaceX's xAI may have reason to worry. The future of AI could be smaller, cheaper and far less profitable than investors expect. On June 1, Nvidia made headlines when it revealed a new desktop AI platform that runs on Windows PCs, raising the possibility that the future of AI might not just be in giant data centres. A ‌Stanford University study published ⁠two weeks ⁠earlier suggested the same. It compared small local language models (SLMs) running on desktop PCs with large language models (LLMs) running in data centres.

The researchers tested a range of SLMs, using both PCs and Macs, on 500,000 ​chat requests and 500,000 reasoning tasks. The study found that, on average, these SLMs were as good as or better than LLMs in over 80% of tasks, with success ratios approaching ​100% in sales, management and entertainment applications.


That doesn't mean these SLMs are superior across the board. In the most difficult reasoning tasks, they keep up with LLMs in only about 50% of cases. But that is up from a mere 8% two years ago, showing that the performance gap between SLMs and LLMs is closing fast.

More importantly, ​SLMs are rapidly improving in a metric the researchers call "intelligence per watt," which measures the accuracy of an ⁠SLM on ‌a desktop PC relative to the energy consumed. That measure has improved over five times in the last two years. The result is ​not only that SLMs ​can perform as well or better than LLMs in most cases, but that they can do so while using 50% to ⁠80% less energy - meaning at lower cost.

Trouble for hyperscalers

If this trend continues and SLMs close ​the performance gap with LLMs faster than the market expects, the consequences for the companies driving today's AI boom ​could be severe. A couple of months ago, I argued that LLM hallucinations could undermine the long-term business models of the AI heavyweights.

This Stanford study now implies that LLMs are economically the most viable solution in just one-fifth of current use cases.

If true, this would undermine the lofty valuations that OpenAI and Anthropic hope to achieve in their IPOs - and call into question SpaceX's $2.85 trillion valuation, which is rooted largely in AI hopes.

These firms could enter the race for SLMs by shrinking - or "dumbing down" - their existing models. But the problem is that the most advanced SLMs are open source, meaning they are available for free or at an extremely low cost. Profit ‌margins for LLM providers would thus be much lower in the SLM space.

Moreover, an SLM doesn't need a data centre. If most tasks can be performed at lower cost on a desktop PC, the case for vast data centres packed with expensive GPU, TPU ​and Trainium chips weakens considerably. ​Many of those data centres being built today ⁠may end up being little more than white elephants. If the data centre surge were to come to an end, that could trigger a chain reaction that reverses the AI boom. Growth expectations for hyperscalers would be rolled back, and capital expenditures would be curtailed, which, in turn, would slow growth for chipmakers.

The only companies ​that would likely benefit from such a shift would be desktop computer makers such as Apple - and, potentially, Nvidia, if its new desktop AI platform pans out. In light of this study, Nvidia's foray into desktops looks less like a diversification strategy and more like a hedge to remain relevant, no matter how this technology evolves.

(The views expressed here are those of Joachim Klement, an investment strategist for Panmure Liberum. )