Where Do AI Voice Agents Get Repayment Amounts and Dates Wrong?
Several recurring patterns can create errors in AI-powered collections calls.
The original monitoring framework identifies five important categories.
| Error Pattern | What the Borrower Says | Potential Failure |
|---|
| Relative dates | "Next Friday," "end of the month," or "after payday" | The agent assumes a date that the borrower did not intend |
| Date formats | "The third of the fourth" or "03/04" | The system interprets the month and day incorrectly |
| Split payments | "₹1,000 today and the rest on the 20th" | Only one installment is recorded, or the amounts do not reconcile |
| Missing account fields | "What do I owe including charges?" | The system has missing or stale information, and the agent generates an unsupported amount |
| Multilingual number expressions | Amounts spoken in Hindi, Tamil, Arabic, or other languages | Speech recognition or number normalization produces the wrong value |
1. Relative Payment Dates
Borrowers frequently use informal date expressions.
Examples include:
- "I'll pay next Friday."
- "I can pay at the end of the month."
- "My salary comes next week."
- "I'll make the payment after payday."
These statements are not always precise.
For example, "next Friday" may mean different dates depending on when the conversation occurs and how the borrower uses the expression.
An AI agent should clarify ambiguous dates rather than silently selecting one.
2. Date Format Confusion
Different markets represent dates differently.
For example, 03/04 can mean March 4 in a US-style format or April 3 in a day-first format.
India commonly uses day-month-year formats, but systems may still process timestamps or date strings differently.
For NBFCs, the system should normalize dates into an unambiguous internal representation while confirming a human-readable date with the borrower.
3. Split Payments and Partial Commitments
A borrower may agree to:
"₹2,000 tomorrow and ₹3,000 on the 20th."
This is not one payment of ₹5,000 on a single date.
It is a payment arrangement containing two separate commitments.
The collections system must preserve the amount and date for each installment.
Monitoring should identify cases where the AI agent or backend records only the first payment or incorrectly combines both amounts.
4. Missing or Stale Account Information
An AI voice agent may attempt to answer a repayment question without having the necessary account information.
For example:
Borrower: "How much do I owe including late fees?"
If the authorized account service does not return a reliable figure, the agent should not invent a plausible amount.
A safer response is:
"I can't confirm the exact amount right now. I can arrange for the amount to be verified before you make a payment."
The specific wording and next step should follow the NBFC's approved workflow.
5. Multilingual Numbers and Speech Recognition
Number recognition is particularly important for collections agents operating across multiple languages.
In Hindi and Urdu, expressions such as dedh and dhaai mean one and a half and two and a half.
For example, dhaai hazaar means 2,500.
In Arabic and Afrikaans, compound number constructions may express units before tens. A poorly designed speech-processing system could transpose or misinterpret values.
Swahili time expressions can also differ from conventional English clock references, creating potential confusion in callback scheduling.
For Indian NBFCs, similar validation is important across Hindi, Tamil, Telugu, Kannada, Malayalam, Marathi, Bengali, and other supported languages.
The risk is not limited to non-English calls.
Even English expressions like "fifteen" and "fifty" can be misheard, especially in noisy environments.
The practical requirement is simple:
Validate amounts and dates used in collections workflows as structured financial information, not just because a transcript sounds fluent.
How Do You Monitor Promises to Pay on Every Live AI Call?
An effective monitoring process shouldn't rely only on the voice agent's self-reported outcome.
Four checks work together.
1. Grounded Amount Checks
A grounded amount check compares what the AI agent says with the authorized account information and relevant tool outputs available during the call.
For example, suppose the loan management system reports:
- Overdue amount: ₹8,500.
- Next scheduled installment: ₹3,000.
- Authorized settlement amount: Not available.
If the AI agent tells the borrower the settlement amount is ₹6,000 without an authorized source, flag that statement.
Not every spoken amount must equal the total overdue balance. A borrower may propose an authorized partial payment.
The check should therefore distinguish quoted liabilities, proposed payments, payment-plan components, and approved concessions.
It should also preserve evidence of which account value was available when the statement was made.
If a required account field is empty, the agent should ask for verification rather than produce an unsupported figure.
The original Vattara test plan identifies this class of issue as grounded amount validation.
2. Date Normalization
Relative dates should be converted into explicit calendar dates using the call date, applicable time zone, and borrower confirmation.
For example:
Borrower: "I'll pay next Friday."
Agent: "To confirm, do you mean Friday, November 6?"
Borrower: "Yes."
The system can then record November 6 as the agreed date.
If the borrower replies that they meant a different Friday, the agent should clarify the arrangement before logging it.
Monitoring should compare the confirmed date with the saved commitment and any applicable lender policy restrictions.
3. Repayment Read-Back
Before ending the call, the AI agent should confirm the agreed arrangement.
For example:
"To confirm, you have agreed to pay ₹2,500 on November 6 using your preferred payment method. Is that correct?"
A strong read-back check verifies that:
- The agent repeated the correct amount.
- The date was explicit.
- The borrower confirmed the arrangement.
- The recorded promise matched the confirmed commitment.
- Multiple payment components were represented correctly where applicable.
The original Vattara test framework includes read-back validation as a separate evaluation criterion.
Read-back is useful operational evidence, but it should not be described as a universally mandated regulatory script.
4. Affordability and Payment Pressure
A recorded promise does not always represent a suitable arrangement.
If a borrower says they cannot afford the requested amount, the agent should respond according to the lender's approved assistance or escalation process.
An agent should not treat every conversation as an opportunity to secure a payment commitment at any cost.
Monitor whether the agent ignored an explicit affordability concern, applied inappropriate pressure, or failed to offer authorized assistance.
In jurisdictions such as the UK and Australia, applicable rules and guidance place particular emphasis on appropriate treatment of borrowers experiencing repayment difficulties.
For NBFCs in India, repayment conversations should also respect applicable borrower-protection requirements and internal recovery policies.
These four checks—grounded amounts, normalized dates, read-back, and appropriate borrower handling—provide a practical foundation for promise-to-pay monitoring.
Why Can't Manual QA Sampling Catch Every Payment Amount Error?
Traditional collections QA often reviews whether an agent followed the required script and recorded the correct call outcome.
It may not verify every amount or date against the loan management system.
Consider an illustrative portfolio.
Assume 30,000 calls per month result in payment promises.
A QA team reviews 1.5% of those calls, or 450 conversations.
Now assume 1% of promises contain an incorrect amount or date.
That represents 300 affected promises per month.
A random sample of 450 calls would contain, on average, about four or five affected promises.
Most affected calls would fall outside the manual sample.
And even when a problematic call is selected, the reviewer may need to open the corresponding loan account and compare multiple records to identify the discrepancy.
A scorecard asking only whether the agent confirmed the arrangement may mark the conversation as successful.
A stronger approach evaluates whether the agent accurately confirmed the arrangement.
Production observability can automate parts of this comparison at scale, provided the system can access the necessary authorized records and evaluation evidence.
Human QA remains important for calibration, disputes, sensitive cases, and investigating incorrect automated judgments.
What Is the Financial Impact of Incorrect Promise-to-Pay Records?
The financial impact depends on call volume, average commitment amount, error frequency, and whether those errors affect repayment.
Consider the original illustrative model adapted for an Indian NBFC.
Assumptions:
| Input | Monthly Value |
|---|
| Promises to pay | 20,000 |
| Average promise amount | ₹5,000 |
| Promises containing an amount or date error | 2% |
| Share of erroneous promises that fail because of the error | 50% |
The calculations are:
Incorrect promises: 20,000 × 2% = 400.
Promises broken because of the error: 400 × 50% = 200.
Illustrative payment value affected: 200 × ₹5,000 = ₹10,00,000.
Under these assumptions, ₹10 lakh in payment commitments could be delayed or otherwise affected in a month.
This is not a forecast of permanent losses. Some payments may be collected later, while others may have failed for unrelated reasons.
The model also excludes follow-up costs, complaint handling, additional operational work, and potential delinquency effects.
It shows how small data-quality error rates can become financially meaningful at scale.
NBFCs should replace these assumptions with actual portfolio data, error frequencies, verified root causes, and repayment outcomes.
Which Promise-to-Pay KPIs Should NBFCs Track?
Tracking the promise-to-pay rate alone does not accurately reflect collections quality.
NBFCs should also track whether commitments are fulfilled, whether recorded details match the conversation, and why promises fail.
| KPI | Definition | How to Measure |
|---|
| Promise-to-pay rate | Eligible verified borrower contacts resulting in a valid promise | Confirm the commitment and its recorded disposition |
| Promise-kept rate | Promises fulfilled on time divided by promises due | Join commitment records with verified payment data |
| Amount mismatch rate | Promise records containing inconsistencies between agreed, confirmed, or logged amounts | Compare call evidence with structured promise and account data |
| Date mismatch rate | Promises where the confirmed date differs from the logged date | Normalize dates and compare recorded values |
| Read-back rate | Promises with a completed and accurate borrower confirmation | Evaluate the read-back and borrower response |
| Broken promise causes | Reasons commitments were not fulfilled | Combine call analysis, payment systems, and follow-up evidence |
| Partial-payment accuracy | Installment plans correctly recorded | Validate the amount and date of every payment component |
| Unsupported amount rate | Calls in which the agent stated an unauthorized or ungrounded financial amount | Compare spoken values against authorized sources |
Promise-to-Pay Rate
Promise-to-pay rate measures the proportion of eligible right-party contacts that result in valid payment commitments.
Formula:
Promise-to-Pay Rate = (Valid PTP Commitments / Eligible Right-Party Contacts) × 100
The institution should specify whether it counts promises, calls, or unique accounts.
Promise-Kept Rate
Promise-kept rate measures the share of promises due during a defined period that were fulfilled according to the agreed terms.
Formula:
Promise-Kept Rate = (Promises Fulfilled on Time / Promises Due) × 100
Define how partial payments, grace periods, reversals, and revised commitments are treated.
Amount Mismatch Rate
The amount mismatch rate measures the share of monitored promises with a material inconsistency in the payment amount.
Formula:
Amount Mismatch Rate = (Promises with Confirmed Amount Mismatches / Promises Evaluated) × 100
Do not count a valid partial payment as a mismatch simply because it is lower than the outstanding balance.
Date Mismatch Rate
Date mismatch rate measures whether the saved commitment date differs from the borrower-confirmed date.
Formula:
Date Mismatch Rate = (Promises with Confirmed Date Mismatches / Promises Evaluated) × 100
Read-Back Accuracy Rate
Read-back accuracy measures whether the agent correctly repeated the agreed amount and date and obtained the required confirmation.
Formula:
Read-Back Accuracy Rate = (Accurately Confirmed Promises / Promises Requiring Read-Back) × 100
Broken Promise Causes
When a commitment isn't fulfilled, investigate the reason instead of automatically attributing it to borrower unwillingness.
Potential causes include:
- Incorrect amount.
- Incorrect date.
- Borrower financial hardship.
- Incomplete or missing confirmation.
- Payment channel failure.
- Promise not logged correctly.
- Payment made but not reconciled.
- Other operational or borrower-related factors.
You can't infer every cause from the original call. Some require downstream records, additional borrower contact, or human investigation.
Why Should Promise-to-Pay Rate and Promise-Kept Rate Be Monitored Together?
A rising promise-to-pay rate may appear to indicate improving collections performance.
But the interpretation changes if the promise-kept rate declines at the same time.
Consider two hypothetical AI agent versions.
| Metric | Version A | Version B |
|---|
| Promise-to-pay rate | 35% | 45% |
| Promise-kept rate | 75% | 55% |
| Amount mismatch rate | 0.5% | 2.5% |
Version B captures more commitments, but fewer are fulfilled proportionally, and more contain amount inconsistencies.
This may indicate problems with verification, read-back, borrower understanding, affordability handling, or data accuracy.
It does not prove that the agent pressured borrowers into unrealistic promises.
The NBFC should investigate the call evidence, relevant customer segments, and downstream payment outcomes before determining the cause.
Promise-to-pay rate measures commitment capture. Promise-kept rate measures fulfillment. Amount and date accuracy help establish whether the commitment was recorded correctly.
All three are needed for a meaningful assessment.
How Does Voice Agent Observability Improve Promise-to-Pay Monitoring?
Pre-launch testing can verify whether an AI agent handles expected payment scenarios correctly.
It can test missing balances, relative dates, split payments, and difficult borrower responses.
But real collections calls add more variation.
A borrower may change the amount mid-conversation, request a payment date outside the normal policy window, speak a regional language, or challenge a late fee.
Production observability provides visibility into these interactions after deployment.
A practical monitoring workflow has five stages.
1. Capture the Conversation and Relevant Events
Collect permitted conversation records, timestamps, tool calls, and relevant operational events.
2. Extract Financial Commitments
Identify potential payment promises, amounts, dates, conditions, and borrower confirmations.
3. Compare Against Authorized Records
Check spoken amounts, available account information, and logged promises.
Distinguish the loan balance from the amount the borrower agreed to pay.
4. Flag Inconsistencies
Identify unsupported figures, ambiguous dates, incomplete read-backs, split-payment errors, and mismatched records.
5. Investigate and Improve
Route relevant failures to operations, QA, or engineering teams.
Use de-identified or synthetic reproductions of recurring incidents to expand pre-launch regression testing.
This creates a continuous quality process in which real production failures inform future evaluations.
How Does Vattara Observe Support Collections Call Monitoring?
Vattara Observe provides production observability capabilities for AI voice agents, helping teams evaluate live conversations and investigate performance issues.
For NBFC collections, the monitoring framework can be organized around:
Payment information accuracy: Evaluating whether the agent communicates amounts and dates accurately against available authorized data.
Promise confirmation: Checking whether repayment commitments are clearly understood, confirmed, and recorded.
Conversation quality: Monitoring interruptions, latency, repeated questions, and unsuccessful interactions.
Borrower handling: Evaluating configured rules for appropriate responses to financial hardship, disputes, or escalation requests.
Operational performance: Connecting conversation-level findings with business metrics and workflow outcomes where supported by the deployment.
The original Vattara product framework also describes call-level evidence, business objective tracking, and pace alerts for payment commitments.
Exact data integrations, automated matching logic, and alert capabilities should be confirmed for the customer's deployment.
The objective is not to replace the NBFC's loan management or payment system.
It is to add conversation-level evidence that helps explain whether the AI agent captured the right commitment and why problems occur.
Frequently Asked Questions
What Is Promise-to-Pay Tracking in Collections?
Promise-to-pay tracking records repayment commitments and checks whether they are fulfilled. For AI collections calls, it should also verify that the borrower-agreed amount and date match the agent's confirmation and the recorded promise.
Why Do AI Voice Agents Sometimes Invent Repayment Amounts?
An AI voice agent may generate an unsupported figure when account data is missing, stale, incorrectly retrieved, or misinterpreted. Because language models can produce plausible responses without verified financial data, ground amounts in authorized account records and tool outputs.
How Should AI Agents Handle Relative Dates Like "Next Friday"?
The agent should clarify the intended date, express it as an unambiguous calendar date, and obtain borrower confirmation before saving the commitment. The monitoring system should compare that confirmation with the logged date using the appropriate time zone.
What Is a Good Promise-Kept Rate for NBFCs?
No single reliable benchmark exists across all NBFC portfolios, loan products, delinquency stages, and customer segments. Institutions should compare similar portfolios, establish internal baselines, and analyze changes alongside amount accuracy, date accuracy, and borrower handling.
Does Regulation require a Repayment Read-Back?
No universal regulation prescribes one exact read-back script for all collection calls. However, accurate, well-documented repayment arrangements support sound collections operations and help reduce misleading or incorrect communications. Specific obligations depend on the applicable jurisdiction and loan product.
Can AI Observability Detect Incorrect Payment Amounts Automatically?
Yes, where the monitoring system can compare amounts stated during conversations with authorized account records or tool outputs. Accuracy depends on reliable data integration, correctly classifying what the amount means, and appropriately evaluating the conversation context.
What Is the Difference Between Promise-to-Pay Rate and Promise-Kept Rate?
The promise-to-pay rate measures how often eligible borrower contacts result in valid repayment commitments. The promise-kept rate measures the proportion of those commitments due during a defined period that were actually fulfilled.
Why Should NBFCs Track Broken Promise Causes?
Broken promise analysis helps distinguish borrower nonpayment from operational errors such as incorrect dates, missing confirmations, payment processing failures, and account reconciliation problems. Understanding those causes supports better collection decisions and agent improvements.
Conclusion: A Recorded Promise Is Not Always an Accurate Promise
For NBFCs deploying AI voice agents, a successful collections call should mean more than populating a promise-to-pay field.
The borrower must understand the arrangement. The AI agent must communicate accurate amounts and dates. The recorded commitment must match what was agreed, and downstream systems must correctly track whether the payment occurred.
An AI collections agent can sound confident while communicating an incorrect figure.
It can mark a call as successful while storing the wrong payment date.
It can capture more promises without improving actual collections.
That is why promise-to-pay monitoring should connect conversation quality, financial-data accuracy, commitment confirmation, and payment outcomes.
Pre-launch testing helps identify predictable errors. Production observability helps teams find failures during real borrower interactions and understand their operational impact.
For NBFCs, the goal is not simply to capture more repayment promises. It is to capture accurate, appropriate commitments and measure what happens afterward.
Monitor Repayment Accuracy Across Live AI Collections Calls
A traditional collections dashboard shows how many promises were recorded. Voice agent observability helps investigate whether those promises were captured correctly.
Explore how Vattara Observe supports production conversation monitoring, AI voice agent evaluation, and reliability investigations for collections operations.
Explore Vattara for Collections
