A banking voice AI vendor should be evaluated with actual call evidence, not slides alone. Use these 50 questions to check identity verification, borrower privacy, data reliability, language handling, telephony, recovery conduct and post-launch monitoring. Each answer should be supported by a test call, configuration record or auditable workflow. Print this and take it into your next vendor demo. For every answer, ask to hear a call, not to see a slide. The questions come from what broke when we tested collections voice agents against Indian lending rules in October 2026.
What a Strong Vendor Answer Should Include
For every question, ask for evidence: an unedited demonstration, a test result with the scenario and denominator, a configuration or log extract, and the named owner of the control. An assurance such as “the prompt handles this” is not equivalent to verified behavior. Treat legal obligations and additional recommended controls as separate rows in your scorecard.
RBI-Related Compliance and Conduct Questions
- Q1. Does the agent say it is an automated assistant in its first sentence? What does it say when asked "am I talking to a person"?
- Q2. Does it name both the brand the borrower knows and the registered entity, and give the RBI registration number when asked?
- Q3. If an account is assigned to a recovery agency, does the agent check that the borrower was notified before it discusses dues? What does it do if the notice is missing?
- Q4. Before verification, can the agent say "loan", "EMI", "overdue" or any amount? Show me a call where someone tried to make it.
- Q5. When a spouse, relative or co-borrower answers, what exactly does the agent say? Does it confirm that a loan exists?
- Q6. What does the agent say when the borrower asks it to stop calling, and where is that request recorded?
- Q7. Can the agent mention repossession, auction, legal action or police? Under what record conditions, and in which words?
- Q8. What does the agent do when a borrower describes hardship: a lost job, an illness, a death in the family?
- Q9. When a borrower complains, does the agent give a reference number, the response timeline and the Ombudsman route? Where does the reference number come from?
- Q10. Can the agent agree to a waiver, settlement, extension or new date? If not, what sentence does it use when asked, and how do you know it never says "ho jayega"?
- Q11. Does the agent keep the same tone when the borrower is abusive, or says the call is being recorded?
- Q12. Does the agent ever ask about, or acknowledge, religion, caste or community?
Borrower Verification, Data Security and Privacy
- Q13. Which two identity fields does the agent verify, and against which source?
- Q14. What happens when the borrower opens with a demand, or with "I already paid", before verification?
- Q15. Does any real borrower data leave our environment during testing? If so, where does it go, and under what agreement?
- Q16. Where are recordings and transcripts stored, for how long, and in which country?
- Q17. What does the agent do with personal information the borrower volunteers and the call does not need?
- Q18. How is consent captured for a mandate or a bureau pull: the exact words, and what counts as a yes? Is "hmm" a yes?
- Q19. When the agent reads our systems through a tool, what happens if the tool fails or times out mid-call?
Multilingual Voice AI Testing and Speech Recognition
- Q20. Which languages has the agent been tested in, with how many calls in each? Supported is not the same as tested.
- Q21. Show me one failed call in each language. If there are none, how was the agent tested?
- Q22. In your tests, was the call language chosen by the test, or inherited from whatever the customer record held?
- Q23. What does the agent say on its first turn to a borrower who speaks a language other than its default?
- Q24. How does it handle code-switching inside one sentence: Hinglish, Tanglish, an English figure inside a Marathi sentence?
- Q25. Which regional registers were tested: Bhojpuri-influenced Hindi, Marwari, rural Marathi, Deccani? Which were not?
- Q26. How does speech recognition handle names written as initials, common in South Indian records?
- Q27. How does the agent perform on a speakerphone in a shop, on a two-wheeler, on a poor network?
- Q28. Is speech recognition accuracy on amounts measured separately from overall word accuracy?
- Q29. If the agent and the test caller run on the same speech vendor, how do you know the test is independent?
Payment Accuracy, Promises to Pay and Interruptions
Collections heads: this is your section.
- Q30. When the borrower makes a promise to pay, does the agent read back the amount in digits and the date with its weekday, and wait for a confirmation?
- Q31. What happens to an amount of "paune do lakh" in your pipeline? Show me what was heard, what was stored and what was read back.
- Q32. What does the agent say when the overdue amount field is blank? Show me the call.
- Q33. When the borrower disputes a figure, does the agent repeat its own figure, or say the team will check?
- Q34. How does the agent convert relative dates such as "next Monday", "after salary" or "after market day"? Tested in which languages?
- Q35. When a borrower ties payment to an event, a refund or a harvest, what does the agent record?
- Q36. Can the borrower interrupt mid-sentence, and does the agent stop and listen?
- Q37. Does the agent ask again for information the borrower has already given? How often, and how is that measured?
- Q38. Can a promise to pay be logged against an amount that is not on record?
Telephony Reliability and Calling Hours
- Q39. Where is the 8 am to 7 pm calling window enforced: in the agent's prompt, or in the dialer?
- Q40. What does the retry logic do with a call that first fails at 6:40 pm? Can a retry land after 7 pm?
- Q41. Which number series do the calls come from, and is it registered for this purpose?
- Q42. What happens on voicemail, a busy tone, or a line that drops mid-sentence?
- Q43. What is the agent's response time per turn at the 95th percentile, measured on our telephony rather than in a demo?
- Q44. At how many concurrent calls has the agent been tested, and does its behaviour change under load?
Voice Agent Testing, Monitoring and Audit Evidence
- Q45. How many test calls ran before go-live, per scenario family, and who scored them?
- Q46. Which tests rerun automatically when the prompt, the model or the voice changes?
- Q47. Are results reported per scenario and per language, or only as an average?
- Q48. Who reads the transcripts of failed calls, and how soon after the call?
- Q49. How are live calls checked against the same rules used in testing, and what triggers an alert?
- Q50. Can you give us the scored transcripts of ten failed calls from your own testing? A vendor with none has not tested.
How to Use This Voice AI Vendor Evaluation Checklist
In October 2026 we tested collections, servicing, reminder and disbursement voice agents against these families. A default agent invented an overdue amount on 15 of 18 calls where the field was blank in our 9 to 11 October voice run, confirmed a loan's existence to a borrower's relative, and ignored four requests to stop calling. A hardened agent did far better and still invented a figure under pressure. Every question above maps to something we saw break, or to something we could not test and think you should.
If you want these questions run against your own agent, in your languages, with the transcripts, talk to us.
Editorial note: All experiments described here are synthetic unless expressly identified otherwise. Confirm regulatory sources, product availability, technical specifics and source transcripts before publication.
