Sam Altman: AI model training expensive, but doable

OpenAI CEO Sam Altman, on Wednesday, highlighted the evolving economics of artificial intelligence (AI), noting that while training frontier models remains expensive, breakthroughs in distillation techniques are making AI more accessible.

Altman made these remarks in New Delhi during a conversation with IT minister Ashwini Vaishnaw. He also sought to clarify his past remarks at the The Economic Times Conversations in 2023 where he had suggested that Indian startups would struggle to build foundational models due to high compute costs. Altman said that his comments were “taken out of context.”

“That was a very specific time when there was a certain scaling thing where I thought, you know, and I still think to stay on that frontier of pre-trained models is expensive,” he said.


However, the industry's trajectory has since shifted, thanks to significant breakthroughs in distillation techniques. “We learned a lot to do small models. And these reasoning models in particular can be, it's not cheap. It's still expensive to train them, but it's doable,” Altman added.

This progress, he said, will “lead to an explosion of really great creativity.”

Altman highlighted two key trends shaping the AI industry. On one hand, he acknowledged that pushing the boundaries of AI will continue to demand rising investments. However, he argued that “the returns to increase in intelligence are exponential in terms of the economic value.” At the same time, he pointed to a contrasting trend, stating, “The cost for a given unit of intelligence one year later seems to fall by about 10x.”

The reduction in the cost of AI models is seen as extraordinary, only creating more accessible opportunities for businesses and industries worldwide.

Despite the reduction in model costs, Altman clarified that this doesn't imply the world will need less AI hardware. Instead, as costs decrease, Altman said, the potential applications for AI hardware expand dramatically.