OpenAI's AI agents ‘went rogue’, targeted US government websites without lab’s knowledge: Report

AI agents developed by OpenAI targeted websites belonging to the US Education Department, Commerce Department and Securities and Exchange Commission this summer without the company’s knowledge, according to a New York Times report citing security researchers and a person familiar with the incidents.

The incidents involving the Commerce Department and the SEC were confirmed by OpenAI, which said it was continuing to investigate the episodes, as per the report.

The company has launched a broader review of actions taken by its models during training and evaluation, including instances in which agents interacted with third-party websites beyond their assigned tasks or intended methods.

Government websites targeted

OpenAI CEO Sam Altman said Friday that the company was reviewing use cases involving agents with internet access, but acknowledged that the process had not moved as quickly as the company would have liked.

"There is an extensive and ongoing review related to our agents' use of internet access during training and evaluation. We've been publishing summaries at the link below and will continue to. We have not been as fast as we would have liked but we are trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs, and working with impacted organizations," Altman said in a post on X.

He said OpenAI was prioritising cases based on severity and adding resources to the review.

The New York Times reported that the government website incidents were among episodes in which OpenAI's models interacted with third-party websites without the lab's knowledge. The report cited security researchers and a person familiar with the incidents.

OpenAI said the vast majority of actions reviewed so far involved routine research tasks, such as accessing publicly available web content to answer questions.

Its investigation is focused on cases where agents interacted with third-party websites in ways that went beyond their assigned tasks or intended methods. OpenAI said most cases identified so far were of lower severity, with limited or no evidence of meaningful impact on the third-party services.

OpenAI expands agent review

The company said it was notifying third parties when its models may have bypassed security controls, impaired the availability of an online service or otherwise negatively affected third-party websites.

"Based on our review to date, we have notified dozens of third parties using the criteria above. Our review of past activity is ongoing and will require significant time and resources. We will notify additional third parties as that work continues," OpenAI said.

Altman said the company was trying to balance transparency with the need to understand activity across what he described as petabytes of agent logs and to coordinate with affected organisations.

He said the company would be as transparent as possible, subject to cases involving vulnerabilities discovered in other companies where disclosure would be their decision.

The New York Times report said the episodes involving the Commerce Department and SEC had been confirmed by OpenAI, while the company continued its investigation.

OpenAI said a notification to a third party should not automatically be interpreted as notice of a significant security incident. It said it would continue sharing relevant findings with affected entities and provide technical information to support their reviews.