OpenAI, Anthropic model tests reveal more ‘unsanctioned’ actions

Artificial intelligence models developed by OpenAI and Anthropic PBC carried out “unsanctioned” actions — including hacking a website and attempting to inject harmful code into software during safety testing — reinforcing fears that neither the creators nor seasoned researchers of these systems can predict their actions in testing.

The UK government’s AI Security Institute, established in 2023 to evaluate the safety of cutting-edge AI models, said Tuesday that both Anthropic’s Mythos 5 and OpenAI’s GPT-5.6-Sol models had both “engaged in sustained, potentially harmful activity directed at real people and organizations” during evaluations. The institute intentionally allowed the models internet access and used them without certain safety filters to test their capabilities.

“Even under test conditions, this incident is significant: It is the first time we have seen risks around autonomy and deception manifest this clearly in the real world,” the institute said in a post on the social media platform X.


In one instance, the testing organization said, Mythos 5 attempted to add harmful code to an open-source software project on GitHub. It went as far as to create fake identities in an effort to get its code approved. “A human maintainer caught and refused to approve the malicious code,” the group wrote.

Over the past two weeks, both OpenAI and Anthropic have publicly acknowledged that they’ve collectively breached the systems of multiple institutions including Hugging Face Inc. inadvertently while testing their models. The latest disclosures serve as fresh evidence that AI agents are capable of acting autonomously in ways that even researchers trained to root out vulnerabilities in the technology can no longer anticipate, underscoring the need for both more rigorous safety screening and more foolproof testing environments.

The UK’s AI security institute said Anthropic’s Mythos 5 model carried out 17 of the 19 “autonomous, unsanctioned actions taken on the internet“ that it detected.

Some US government leaders have called for more oversight of the technology in response to the breaches. Last Tuesday, more than 1,100 AI industry workers signed a petition pushing for a regulatory mechanism that would “deliberately pace” AI technology and prevent it from advancing too quickly.

Anthropic said on X that it’s working with the UK security institute to “gather more details of the incident as we conduct our own investigation.”

ChatGPT maker OpenAI separately flagged in a blog post that yet another security incident occurred during the testing of one of its models with Irregular, an external cybersecurity firm.

In this incident, OpenAI’s models were subject to a so-called capture the flag test in which they were tasked with finding information hidden in a simulated environment. The models took advantage of a “misconfiguration” in the testing environment to connect to the internet and hack the website of an unidentified institution, the company said.

The breach occurred when OpenAI models were undergoing the same Irregular evaluation that resulted in Anthropic’s models hacking three organizations, a person familiar with the matter said, asking not to be identified because the information isn’t public. Anthropic disclosed those breaches last week.

An Irregular spokesperson declined to comment.

Two weeks ago, OpenAI disclosed that its models were behind an unprecedented hack against the startup Hugging Face. In that incident, the models exploited a vulnerability to “escape” their sandbox testing environment and connect to the internet, at which point it breached Hugging Face’s system, which hosts AI models and datasets.