Governance, not models, is the biggest obstacle to enterprise: Red Hat’s Vincent Caldeira

The biggest obstacle to enterprise artificial intelligence (AI) is not the underlying model but the governance behind it, according to Red Hat’s APAC chief technology officer (CTO) Vincent Caldeira, as companies move generative AI adoption from small experiments into systems for employees and customers.

Red Hat is a US-based software company and an IBM subsidiary.

“The biggest obstacle to enterprise AI is no longer the model. For me, it’s governance,” Caldeira told ET AI.


He said enterprises have become increasingly focused on two issues as they put AI into production – cost and governance. A proof of concept for five users may be manageable, he said, but the risks and economics change when it reaches a million customers.

As per Caldeira, governance extends beyond conventional cybersecurity or just preventing a system from being hacked. It also covers the consequences of the decisions of an AI system.

Also read: Red Hat sees an opening as Indian enterprises turn to AI sovereignty

“Can it provide the wrong recommendation to your staff or can it insult your customer? How do you actually govern the outcome of an AI system?” he questioned.

The issue has particular relevance for Red Hat in India, where organisations operating major financial, telecom and transport systems use its technologies. The company counts the National Stock Exchange, BSE, Unique Identification Authority of India, Bharti Airtel, Jio Platforms, Tata Motors, IndiGo, State Bank of India, Indian Bank, Bank of India and Unity Small Finance Bank among its customers.

Two years ago, companies could experiment with generative AI using open-source tools and a do-it-yourself approach, Caldeira added. As those projects scale, businesses are recognising the need for platforms addressing observability, identity, access controls and guardrails.

He said questions around agentic security have become much more prominent over the past six months. Enterprises are now asking, “How do we evaluate an agentic AI system and how do we … judge its readiness for a production run?”

He added that organisations were underestimating the difficulty of securing their autonomous systems, particularly when they are deployed inside regulated institutions.

The industry does not yet have a platform that can make an AI agent completely secure out of the box, according to Caldeira. He said that while standards and individual technologies were emerging across different layers of agentic security, they have not been integrated into a comprehensive platform so far.

Caldeira said vendors claiming to have already solved the problem were overstating their capabilities.

“I keep telling customers that if a vendor says it has an out-of-the-box solution for a secure agent, the salesperson is lying. It does not exist today,” he added.

Also read: AI security emerging as separate budget line for Indian enterprises: Palo Alto Networks’ Swapna Bapat

Caldeira said Red Hat’s approach includes what it calls evaluation-driven development. Under this, an AI system is repeatedly assessed, as it is built and monitored, until it is considered ready for production. He added that the company was working on real-time guardrails and controls embedded in the underlying cloud-native platform so agentic applications inherit them when deployed.

He described this approach as “shift down,” placing identity, observability, access and other security policies in the underlying platform instead of expecting every application developer to implement them separately.

He also said enterprises should look beyond comparisons between individual models. “The model choice is not the most important choice in your AI system design now. It’s not. We’ve gone beyond that.”

Instead, Caldeira said companies need to evaluate the complete system around the model, including how it retrieves information, calls tools and controls an agent’s actions. He argued that even a powerful frontier model would produce poor results if an enterprise’s retrieval system fed it irrelevant or inaccurate information.

“Suppose I use a cutting-edge frontier model in a system that requires document retrieval. If the document-retrieval process is extremely poor and my agentic workflow begins by retrieving poor information from the enterprise, no level of model capability will fix the problem,” he said.

Also read: The AI pilot era is ending. Now enterprises are worrying about the bill, says Snowflake

“I have seen this frequently. People build very poor RAG systems and then say the model is bad. The model may be entirely capable, but it is being fed terrible or irrelevant data," he added.

Retrieval-Augmented Generation (RAG) is an AI framework that improves large language model (LLM) responses by fetching relevant data from external sources before generating an answer.

Cost, as per Caldeira, is the other major reason enterprise AI projects stall. He said some customers had withdrawn access to coding assistants such as Copilot or Anthropic’s tools after high usage bills began offsetting the productivity improvements.

He said Red Hat believes companies can lower these costs by routing simpler tasks to smaller open-weight models while reserving expensive frontier models for other queries. “We use this internally with our own software development teams. We know that a better approach, which involves intelligently routing coding requests to different types of models, can reduce costs by 80-90%. You can become an efficient token factory that still delivers better coding automation without putting as much pressure on the budget.”