AI kids need adult eyes in the lab

Satya Nadella recently sought a reset in the safety architecture for AI because he doesn't believe the technology can make itself safer as it scales. Sam Altman feels the world must act before it loses critical control over autonomous agents. Anthropic has warned in a filing for a massive IPO that advanced AI could pose catastrophic or existential risks to humanity. AI agents have broken out of controlled environments, acted as coordinated swarms and attacked external platforms. All without human authorisation. The big fear among tech leaders is that recursive self-improvement is not being matched by AI's alignment with human values despite its superhuman capabilities. Although not a Frankenstein moment yet, investors pouring billions of dollars into the IPOs of SpaceX, Anthropic and eventually OpenAI must heed the caution. The kids in the chemistry lab are asking adults to be around. That would mean governments as well as investors.

Safety protocols are straightforward. Slow the pace of development of frontier AI models and have the process audited by external evaluators. And build a global oversight model like that for nuclear testing to ensure safety. Neither course is easy. Companies building AI models with vast amounts of investor money will find it difficult to give up their competitive advantage. If the big boys do agree, they will have to convince governments not to fall behind in the AI arms race. A global response along the lines of the nuclear non-proliferation regime has its limitations. It operates after the genie has left the bottle. Yet, these are the best defence mechanisms we have for a technology that poses existential risks. Its development must be supervised. Voluntary restraint is called for from technology developers, lawmakers and investors.

Controlled development of AI may be less disruptive for society, which will have a longer time to adapt to its capabilities. AI will not be left unsupervised, and humans need time to acquire the required skills.