ET AI Conclave & Awards 2025: “We’re rushing to production without the right scaffolding”

The AI race has made shipping products dangerously easy. With code assistants writing functions in seconds and large language models (LLMs) assembling entire workflows from prompts, product cycles are collapsing. What once took weeks now takes hours. For many organisations, that velocity feels like victory.

But Rahul Chari isn’t so sure.
Speaking at the ET AI Conclave & Awards 2025 in Bengaluru, the PhonePe Founder and CTO offered a sober counterpoint to the speed-first mindset sweeping engineering teams.

“The ability to actually achieve your output results in a very, very short period of time,” he said, “is making it almost a path to get to production as fast as possible without thinking of whether the right scaffolding… is in place or not.”


For a small startup building from scratch, that risk might be manageable. You can experiment, ship, iterate, and rebuild. But for a platform operating at the scale of PhonePe, which handles vast volumes of transactions and users, AI adoption without architecture is a structural liability.

Chari framed the discussion around what he calls “AI-first engineering”. Not AI-as-a-tool or AI-as-a-feature, but AI embedded across engineering and operations.

At PhonePe, that shift is anchored around three goals: dramatically increase the efficiency of a large engineering workforce, take AI-first features to production safely and quickly, and automate productivity across corporate and operational functions, not just technology teams.

The temptation, he acknowledged, is to begin by distributing LLM access and agent tools across the organisation. But that decentralised enthusiasm creates fragmentation (inconsistent authorisation, scattered knowledge bases, uncontrolled token usage and potential security blind spots).

The answer, in his view, is to build the infrastructure layer first.
“You create a singular gateway that then acts as a router for multiple LLMs,” he explained, describing an inference engine and LLM gateway that centralises all model interactions. Whether teams use open-source models, hyperscaler APIs, or proprietary systems, everything flows through one controlled layer.

That gateway enforces quotas, rate limits, authorisation policies and cost governance. It also enables policy-based routing, ensuring that lower-cost models handle batch workflows, while more expensive low-latency models are reserved for time-sensitive use cases. In practical terms, it’s less glamorous than a demo. But it’s the difference between experimentation and sustainable deployment.

Beyond the gateway, Chari advocated building a centralised vector data store and an internal agent marketplace, an ecosystem where reusable AI agents can connect to work collaboration platforms, email, documentation systems and shared knowledge bases. Instead of siloed automation projects, organisations build standardised, shareable workflows with governance baked in.

If this sounds cautious, that’s intentional.
Chari drew a distinction between greenfield AI development and what he called the “mature code problem”. Zero-base projects can rely heavily on prompt-driven builds and minimal human oversight, but most large enterprises operate extensive legacy codebases. In those environments, wholesale replacement isn’t practical.

“Look at every step of your SDLC [Software Development Life Cycle] process in a mature organisation and take an opportunity to target that with smaller solutions,” he advised.

That might mean automating unit test generation, embedding AI into IDEs for inline coverage, fine-tuning models on the best internal reviewers to scale merge request quality, or gradually moving toward automated approvals. The gains come not from replacing engineers but from amplifying them.

“My worry always remains that 99% of the time, everything works really well. The 1% where you may have production issues is something you always have to think about,” Chari observed candidly.

In financial systems at scale, that 1% is not theoretical. If engineers are removed entirely from the loop in pursuit of full automation, mean time to recovery can expand, and the cost of failure multiplies.

His prescription is pragmatic: start with humans in the loop, use AI as a strong enabler, reduce friction, and then automate responsibly. Not the reverse.

Move fast and lean in into AI, Chari summed up. Albeit, with two conditions: build the scaffolding before unleashing AI across the team, and optimise mature systems step by step rather than going zero-base overnight.

AI is here. Its potential is undeniable. But at enterprise scale, the winners won’t be those who ship the fastest proof of concept. They will be the ones who build durable systems that can evolve as models change, absorb shocks when things break, and recover quickly when that inevitable 1% arrives. Because in the AI decade, speed may win headlines. Architecture wins longevity.

The ET AI Conclave & Awards 2025 is driven by BYD and has L&T Finance as NBFC partner; Snowflake as AI data cloud partner; EY as evaluation partner; T-Hub as ecosystem partner; Zoho as technology partner; and Indri as celebration partner. Vahdam India, Vaaree, Natch, and Andamen are the event’s gifting partners.