When AI agents cross the line: Govt assessing whether AI labs should formally disclose autonomous agent intrusions

Mumbai | New Delhi: Government is assessing if there should be formalised disclosure mechanisms for global organisations doing artificial intelligence (AI) research if and when AI agents intrude into the systems of Indian organisations.

Discussions on containing such incidents may form part of the government's planned consultation paper on artificial intelligence regulation which is expected within a month. This follows the latest incidents where AI agents breached Indian organisations' systems without authorisation.

"We are assessing the situation. This time the incident came to light after the AI labs have voluntarily made disclosures since the affected entities did not know about the intrusion. It needs to be considered if there should be formalized disclosure mechanisms for global organizations doing AI research if and when such things happen again," an official said.


Indian regulations require entities to report cyber incidents within six hours.

Read more - No meddling as 'middlemen' in forex conversion by banks: New rules forbid conversion of foreign earnings into rupees to detriment of exporters

Officials said labs have provided complete logs, names of affected targets. They may also provide audit reports of their autonomous testing frameworks. The incidents of AI models accessing Indian systems have taken place over weeks, officials said.

The Indian Computer Emergency Response Team (CERT-In) said on Thursday that agents being evaluated by AI model providers went beyond their assigned tasks and accessed business records, personal data and confidential material.

None of the affected organisations detected the access. The incidents came to light after AI providers reviewed their agents' activities and voluntarily reported them, CERT-In said in an advisory rated critical. It did not identify the providers or affected organisations, or specify how long the access went undetected.

The incidents have raised questions about how developers monitor these systems and report harm when agents act beyond their authorised boundaries.

The broader regulatory exercise is being led by the Ministry of Electronics and Information Technology (MeitY). IT minister Ashwini Vaishnaw said on October 8 that the government would release a consultation paper within a month to lay the foundation for an AI regulatory framework. Its focus would include AI safety, deepfakes and a human-first approach, combining legal safeguards with technological measures.

Vinayak Godse, chief executive of the Data Security Council of India (DSCI), said the incidents reflected the difficulty of keeping pace with increasingly capable AI systems. "Autonomous swarms coordinating in high-dimensional spaces are leading to global instances of AI agents exceeding their remit and accessing information systems unauthorized. This is happening in the US, Australia, and UK. Now, we are seeing the instances in India, as per CERT-In notification," he said.

Godse explained that even developers of the frontier models are struggling to understand this emergent behaviour emanated from rapidly advancing AI. "The technology seems traversing faster than humans and institutions can accumulate experience from it."