High school maths trips Olympiad gold medalist AI models: Google Deepmind CEO answers why

Google Deepmind chief executive Demis Hassabis said that advanced AI models like Gemini can surpass benchmarks like the International Mathematical Olympiad (IMO) but struggle with basic high school maths problems due to inconsistencies.

"The lack of consistency in AI is a major barrier to achieving artificial general intelligence (AGI), " he said on the "Google for Developers" podcast, adding that it is a major roadblock in the journey.

Artificial general intelligence, or AGI, is generally understood as software that has the general cognitive abilities of human beings and can perform any task that a human can.


He also referred to Google CEO Sundar Pichai's description of the current state of AI as "AJI", or artificial jagged intelligence, where systems excel in certain tasks but fail in others.

Road towards AGI

The Deepmind CEO said just increasing data and computing power won't suffice to solve the problem at hand.

He highlighted that rigorous testing and challenging benchmarks can precisely measure an AI model's accurate progress.

"We need better testing and new, more challenging benchmarks to determine precisely what the models excel at and what they don't."

Also Read: AI helps Big Tech score big numbers

Not just Google

ET reported that artificial intelligence (AI) agents, hailed as the "next big thing" by major tech players like Google, OpenAI, and Anthropic, are expected to be a major focus and trend this year.

OpenAI launched Operator, its first AI agent, in January this year, for Pro users across multiple regions, including Australia, Brazil, Canada, India, Japan, Singapore, South Korea, the UK, and most places where ChatGPT is available.

Last October, Anthropic launched an upgraded version of its Claude 3.5 Sonnet model, which can interact with any desktop application. This AI agent can perform desktop-level commands and browse the web to complete tasks.

Also Read: ETtech Explainer | Artificial general intelligence: an enabler or a destroyer