Language support is not the same as demonstrated multilingual reliability. Our synthetic text scenarios revealed differences in date handling, numeric responses and first-turn language matching across six Indian languages. The cross-language findings are text results; associated voice tests were largely in Kannada and do not validate six-language audio performance. We ran the same collections scenarios in six Indian languages. The default agent failed in all six, with a different invented number in each. Both agents got a date wrong in three languages and right in the other three. Language support is a distribution, not a checkbox.
What Should Multilingual Voice AI Testing Measure?
Track the number of eligible tests and failures by language, first-response language match, number normalization, relative-date interpretation and task outcome. For audio, separately measure speech recognition, interruptions and background-noise conditions. Do not transfer success on typed multilingual prompts into a claim about spoken-language accuracy.
How We Tested Agents Across Six Indian Languages
Two collections scenarios in each of six languages, by text, on 9 October 2026, against a default agent and a hardened one. In the first, the borrower offers part payment now and the rest "next Monday". In the second, the borrower asks for a waiver. The customer record had no overdue amount on it. Text isolates what the agent decides.
| Language | Default agent: invented overdue figure | Hardened agent: blank-amount rule | "Next Monday" (correct: 12 October) |
|---|---|---|---|
| Hindi | 10,000 and 12,000 | Held, except once read as zero | Correct |
| Tamil | 5,000 and 12,000 | Held once, failed once: 12,000 | Correct (asked as next Tuesday, 14 October) |
| Marathi | 10,000 and 5,000 | Held | 13 October on both agents |
| Kannada | 8,000 | Held | Correct on default, 13 October on hardened |
| Bengali | 15,000 and 12,000 | Held | 13 October on both agents |
| Telugu | 10,000, then a balance computed on it | Held | Correct |
The same failures reproduced by voice, where every call ran in Kannada.
Three Multilingual Agent Behaviors We Observed
The failure travels across languages; the symptom does not. The default agent invented a figure in every language, anywhere from 5,000 to 15,000 rupees. A Hindi test finds the failure. It does not tell you what your Tamil borrower will hear. Some failures exist only in some languages. "Next Monday" became 13 October in Marathi, Kannada and Bengali, and stayed correct in Hindi, Tamil and Telugu. Same model, same prompt, same day. A Hindi-only test passes this. A Marathi borrower gets a promise logged for the wrong day. The first turn is the weak turn. In three of six Tamil and Marathi conversations, the default agent's first reply came in Hindi. The hardened agent did the same in two of six. Both corrected on the second turn. On a collections call, the first turn is when the borrower decides whether to stay on the line. One fair result: both agents understood fraction words for amounts in every language we tried, including dedh in Hindi, onnara in Tamil, sade teen in Marathi and onduvare in Kannada. The hardened agent read them back in digits; the default agent read them back in words.
How to Test Multilingual Voice AI Beyond a Language Checkbox
The language must be set by the test, not inherited from the data. If a test harness takes the borrower's language from the customer record, a run can keep drawing the same language and never exercise the others. A report that says "tested in 11 languages" can hide exactly that. Results per language, never averaged. An average across six languages would have hidden the date error entirely. Speech, not only text. Text tests what the agent decides. Speech tests whether it heard the right number to decide on: amounts in lakhs, regional number words, an English figure inside a Marathi sentence, a borrower on a speakerphone in a shop.
Questions to Ask About Language Reliability
- Which languages has the agent been tested in, with how many calls each? Can I hear one failed call per language?
- Was the language chosen by the test, or inherited from the customer record?
- Are dates and amounts checked separately in each language?
- What does the agent say on its first turn to a borrower who speaks a language other than its default?
Text and Voice Test Methodology and Limitations
Thirty-five text conversations in six languages on 9 October 2026, including scenarios not shown here; voice calls in Kannada from 9 to 11 October. Synthetic customer and data. The borrower lines were translated from our English scripts and have not yet been reviewed by native speakers; that review comes before any of this is called a benchmark. These are counts, not rates.
Editorial note: All experiments described here are synthetic unless expressly identified otherwise. Confirm regulatory sources, product availability, technical specifics and source transcripts before publication.
