India Inc seeks legal safeguards as agentic AI raises liability risks

Mumbai: Following a series of developments over the past week that have raised concerns over safety and the ability to control increasingly autonomous AI systems, Indian companies deploying them are turning to legal experts to assess potential vicarious liability, contractual safeguards and the allocation of responsibility when they act autonomously.

The companies are seeking clarity on how responsibility should be divided between developers and deployers, what contractual protections are needed and who could be held responsible when an AI agent takes action that no human specifically intended or approved.

AI safety worries rose last week after Anthropic researcher Jacob Coxon quit, warning that AI labs were racing towards self-improving superintelligence, while the company itself reported misuse of its models to try and develop biological weapons.

The concerns escalated further after Anthropic co-founder Dario Amodei called for a slower pace of frontier AI development, warning that autonomous AI agents could pose serious risks.

His call was backed by Elon Musk, Sam Altman and Demis Hassabis.

Early adopters in India Inc feel the issue is becoming more urgent as businesses move beyond using AI for suggestions and analysis to relying on systems that can make important decisions or communicate with others on their own, without human approval at every step.

The issue becomes particularly important as companies are racing to adopt agentic AI - systems that can work autonomously, make decisions and take actions to achieve a broader goal, often with little or no human intervention.

Attreyi Mukherjee, vice president and head of legal at Mahindra Group's electric vehicle business, said that for corporates, AI adoption is in sharp focus right now and in direct sight of top leadership.

Cybersecurity Obligations

"In the absence of sui generis AI legislation in India, compliance requirements are often benchmarked against the graded, risk-based framework prescribed by the EU AI Act," she said.

In many cases, companies have started to revisit older contracts with AI vendors to clearly define responsibility for errors, hallucinations, data breaches or unintended actions by AI agents.

Rohan Bagai, senior partner at law firm AZB & Partners, said contractual allocation of AI risk is rapidly becoming one of the most important parts of enterprise AI procurement.

Historically, technology agreements focused heavily on availability, intellectual property infringement, confidentiality and conventional cybersecurity obligations.

"With agentic AI, customers increasingly need the contract to address an additional layer of risk: who is responsible when the system produces an incorrect output, leaks information, takes an unintended action or interacts with another system unexpectedly," said Bagai.

He added that the provisions that are now attracting particular attention include limitations of liability, indemnities, responsibility for data breaches, security standards, audit rights, use of customer data for model training, intellectual property risks, subcontractors and third-party models and allocation of responsibility for actions autonomously performed by agents. "Rapid evolution of AI is testing the limits of existing legal and regulatory frameworks, particularly as systems become capable of making decisions and taking actions with limited human intervention," said Ruchi Khatlawala, partner at Little & Co.