Google DeepMind's AI safety proposal raises questions over self-regulation

Earlier this week, Google DeepMind CEO Demis Hassabis posted a proposal, 'A Framework for Frontier AI and the Dawning of a New Age'. He wants the US to establish a standards body modelled on Financial Industry Regulatory Authority (Finra), the industry-funded organisation that polices US broker-dealers under Securities and Exchange Commission (SEC).

Hassabis proposes frontier labs share their most advanced models with this body, voluntarily at first, up to 30 days before release, for testing against cyber, biological and deception risks. Once the testing regime proves itself, participation would stop being voluntary. No frontier model could launch for US users without passing the body's tests first.

The intent deserves respect. Hassabis won the Chemistry Nobel with John M Jumper in 2024 for AI research contributions to protein structure prediction. His worry that competitive pressure is outrunning the field's understanding of its creations is legit and shared by many. The problem is the institution he has proposed to tackle this worry.


The Finra analogy sounds reassuring as it borrows legitimacy of financial regulation without borrowing conditions that make financial regulation work.

Self-regulation is a familiar creature in India, too. Internet and Mobile Association of India's (IAMAI) self-governance codes for digital media are by the industry, for the industry, and of the industry. They work only because a sovereign regulator stands above them with real teeth.

Finra is the same animal. It's a class monitor, a student given a badge, and the arrangement holds because there is a warden, teacher and principal. Remove the principal and you don't have a lighter form of discipline. You have a student with a badge.

Players controlling the referee? Hassabis' proposed AI body, in its opening phase, has no 'principal'. No statute sits above it. Its funding comes from the labs whose models it would judge, because, as Hassabis notes, world-class technical talent and large-scale testing compute are expensive. A referee paid by the players, ruling on the players, with no league above the referee.

Slippery by design There's a second condition financial self-regulation quietly depends on: the object being regulated holds 'still'. A trade is a trade. A security is defined in statute. Finra can write rules because the thing it rules on does not reinvent itself every two quarters.

Frontier AI fails this test by design, and it's not an asset like mutual fund. The proposal itself concedes it, promising that eligibility thresholds will be updated as capabilities grow. The regulated will define, twice a year, what counts as regulatable. In finance, this would be called 'letting the asset manager draft the definition of a security'.

Confidentiality challenge A mutual fund files its strategy with Sebi because publishing it doesn't kill the fund. Its edge lies in execution. For a frontier model, the artefact is the edge. This proposal asks rivals to hand their crown jewels to a shared body, staffed at industry expense, with a board that includes industry representatives, 30 days before launch. Even with the best confidentiality architecture, that is a concentration of pre-release knowledge the industry has never attempted.

Who'll watch the watchmen? Urgency in Hassabis' proposal rests on the claim that artificial general intelligence (AGI) - AI that matches or surpasses human capabilities - is probably a few years away, an event he compares to an Industrial Revolution arriving at 10x scale and speed. Perhaps. But AGI has no standard definition or an agreed-upon test, and has its goalpost moved every time a milestone has been crossed. Not to mention no agreement found yet on what 'general intelligence' is.

Machines producing novel text and passing benchmarks tell us that capabilities are compounding. But it doesn't tell us of a singular threshold approaching on a knowable date. If the countdown is wrong, we will have constructed a permanent gatekeeper for a temporary panic.

The demonstrable risks Hassabis names are real and need no AGI premise to justify testing. The honest sequence runs in the opposite direction to his: statute first, scope narrowed to harms we can define and measure, funding insulated from the funded first. Only then an industry body to execute under supervision.

The question Hassabis never answers is the oldest one in regulation: who will watch the watchmen? Until someone answers it in statute rather than in goodwill, a Finra-like body for AI is not a safety framework. It's the industry marking its own homework, 30 days before the exam.