Blue Machines AI unveils speech-to-text model built for India's BFSI sector

Blue Machines AI, an agentic customer experience platform for enterprises, on Monday (September 7) launched Aurora, a multilingual speech-to-text model designed for banking, financial services and insurance (BFSI) applications in India.

The company said Aurora is designed for real-time financial conversations where customers may switch between languages, combine English financial terms with regional languages, or communicate over noisy and low-bandwidth telephone connections.

In internal benchmarking on representative BFSI datasets, Aurora recorded a Semantic Word Error Rate (WER) of 1.51% for English, 2.43% for Hindi BFSI conversations and 5.52% across multilingual speech, according to the company.


The model also recorded a BFSI Entity Error Rate of 4.23% for information including monetary amounts, interest rates, policy numbers, account references and transaction IDs. Blue Machines AI said it evaluated Aurora against leading speech-to-text models using consistent audio inputs and scoring methodologies.

Also Read: Voice AI startup Navana ai raises Rs 40 crore from Ronnie Screwvala, others

The datasets covered banking, lending, insurance, collections and customer-service conversations, including Indian English, Hindi, Hinglish, multilingual and code-mixed speech, regional pronunciation patterns, background noise and telephony audio.

Unlike general-purpose speech-to-text models, Aurora has been trained on BFSI-specific terminology, including EMIs, outstanding amounts, foreclosure charges, disbursals, KYC, premiums, SIPs, NAVs, policy numbers and transaction IDs.

The company said the model is also designed to recognise numbers, currencies, percentages and financial identifiers that can be used in downstream financial workflows.

“Aurora reflects our commitment to building sovereign AI infrastructure for Indian enterprises,” said Nirmit Parikh, founder and CEO, Blue Machines AI. “India’s financial conversations do not happen in a single language or follow a standard script.”

The model uses a cache-aware FastConformer encoder and streaming transducer decoder to process speech incrementally while retaining conversational context, according to Abhishek Ranjan, chief technology officer at Blue Machines AI.

In internal throughput tests, Aurora supported 960 concurrent real-time streams per H100 at a 320-millisecond operating point and 2,400 concurrent streams per H100 at a 1.12-second operating point, the company said.

Blue Machines AI has also developed training pipelines that allow Aurora to be adapted using customer-authorised enterprise data. The company said this can enable the model to learn an institution’s proprietary product names, terminology, geographies, accents and interaction patterns.

Internal evaluations showed that customer-specific retraining resulted in a 40-45% relative reduction in recognition errors compared with the base model on institution-specific datasets, according to the company.

Aurora integrates with Blue Machines AI’s enterprise CX AI platform and can be used across customer journeys including acquisition, onboarding, lending, collections, servicing, insurance, claims and customer support.

The model can be deployed on a managed cloud, within an enterprise virtual private cloud or on-premises, allowing financial institutions to align deployments with their security, data residency and governance requirements, the company said.