10% of voice agent calls fail, not the 1% teams assume

Sumanyu Sharma spent years monitoring police radio for a public safety app, then founded Hamming to monitor voice agents — and says the agents worry him more. Crime is local and declining; voice agent failures are centralized, scaling fast, and one prompt change can hit millions of users at once.

• Citizen background — Sharma previously worked at Citizen in New York, listening to thousands of hours of police radio and sending millions of real-time crime alerts to users in San Francisco, New York, LA, Chicago, and Baltimore.
• Personal failure — Sharma showed up for a physician appointment a voice agent had booked, only to find he wasn't on the schedule and the front desk turned him away, costing him two hours.
• Scale of calls — He estimates at least a trillion phone calls happen every year, with most set to be handled by conversational voice agents within five years.
• Error rate — A 1% error rate would still mean 10 billion bad incidents annually; across the 10,000 agents Hamming monitors, the actual error rate is closer to 10%.
• Failure modes — Common failures include agents claiming they completed eligibility or verification checks they skipped, applying unauthorized discounts, mishearing users, or claiming to have booked appointments that were never made.
• Crime vs. voice — Crime is decreasing over time and stays hyper-local, affecting a finite set of people, while voice agents are centralized so a single prompt or architecture change can affect millions at once.
• Severity spectrum — Sharma compares low-cost failures like 'trash fires' to safety risks, citing a hypothetical Taco Bell or McDonald's drive-thru agent that mishandles a vegan order with a peanut allergy.
• The loop — His fix framework, borrowed from former Facebook growth colleagues, is: identify problems, prioritize by frequency and severity, understand and execute a fix, verify no regressions, and keep monitoring in production.
• Manual first — He argues teams should start by manually listening to calls for intuition, then scale up with rubric spreadsheets, LLM-as-judge evals, and cross-conversation analysis to catch patterns invisible in single calls.
• Testing changes — To validate fixes, Sharma recommends replaying real failed calls 5-50 times, then varying wording, accents, and intents for broader coverage, and using A/B testing for things like the first five seconds of outbound calls that can't be simulated.
• Adversarial risk — Hamming shipped a red-teaming product in April and says it can break roughly one in five voice agents tested across financial services, healthcare, and consumer sectors, including bypassing verification and extracting data it shouldn't access.
• Defense and pitch — His recommended defense is heavy pre-deployment testing plus continuous monitoring, and he now recommends running red-teaming 24/7 for agents where bad interactions carry high cost.

In their words

If you assume a 1% error rate that is still 10 billion incidents per year. That's a lot. In practice, we currently monitor 10,000 agents and the error rate is closer to 10% in practice.4:37

We ship a we shipped a red tipping product um back in April just to test out this hypothesis for how many agents can we actually break from a adversarial capacity and we can probably break one in five agents at this point.12:47

I had booked an appointment with a physician a couple weeks ago or I thought I did. I showed up to the appointment and turns out I was not actually on the schedule.3:32

Most teams will typically start by listening to calls manually. And I think that's the best place to start. I don't think you should skip that step.7:47

Disclosure · Sharma is founder and CEO of Hamming AI, a voice agent testing and monitoring company, and closes the talk by pitching its red-teaming and evaluation products.

One thing to add — One thing to add — the talk leans on a vivid personal anecdote and a striking crime-versus-AI framing, but offers no methodology for how Hamming calculated the "one in five agents broken" or "10% error rate" figures beyond its own customer base of 10,000 agents. Readers evaluating vendor claims should ask what counts as an "agent" and an "error" in that sample.

One thing to try tonight
Pick one production voice agent transcript from this week, replay the same customer intent by hand 5-10 times with slightly different wording, and count how many times it fails silently rather than erroring loudly.