Voice and support agent testing
Your voice agent passed the demo. Now test the 400th call.
Multi-turn conversations fail in ways a single prompt test never sees. A caller interrupts, changes their mind on turn six, and the agent books the wrong slot or promises a refund you don't offer. I build the scenario suites, simulated callers and release gates that catch those failures before your customers do.
Free, no commitment. One call flow, an honest assessment.
Where voice agents break
Turn 7 breaks what turn 2 got right.
Context drifts over long calls. The agent forgets the account number, asks the same question twice or contradicts what it said a minute ago.
Tool calls with real consequences.
Bookings, refunds, CRM updates. One wrong argument in one call costs money and a customer.
Callers don't follow the script.
They interrupt, correct themselves, change their mind or go silent. The demo never tested any of that.
One misheard word cascades.
A transcription error on a name or a number turns into a wrong lookup three steps later, delivered with full confidence.
Escalation at the wrong moment.
The agent keeps trying when it should hand off to a human, or hands off everything and saves you nothing.
Latency and cost creep.
A prompt change adds a second of dead air per turn or doubles tokens per call, and it ships unnoticed.
What I build for voice agents
Stack-agnostic. The tests run against your agent's API or transcripts, whatever you build on.
Scenario suites from real calls
Redacted transcripts turned into a regression set of multi-turn conversations, each with an expected outcome.
Simulated callers
LLM-driven personas (impatient, confused, off-script, adversarial) that run hundreds of conversations against every change.
Per-turn and end-to-end evals
Task completion, tool-call correctness, policy adherence and escalation decisions, graded turn by turn, not only at the end.
Transcription robustness checks
Realistic errors on names, numbers and accents injected into the tests, so you see how the agent recovers.
Release gates in CI
A prompt, model or voice-stack change that drops task completion below your threshold doesn't ship.
Latency and cost budgets
Per-turn latency and cost per call tracked on every run, with a hard limit you choose.
Already have an eval tool? Good. I implement it.
Braintrust, Langfuse and promptfoo are great tools. A tool doesn't decide what to test, build the golden dataset, calibrate the judge against human reviewers or wire the gate into your release process. That's the work I do.
I'm tool-agnostic. If you already use one, I build on top of it. If you don't, I help you pick the one that fits your stack, and everything stays in your account.
Building an eval or observability tool? I'm open to implementation partnerships.
How an engagement works
- 01
Reliability review 路 30 min, free
I look at one AI feature with you and tell you honestly whether I can help. If I can't, I say so.
- 02
Audit 路 2 weeks, fixed price
Failure mode map, baseline eval scores, risk report, fix plan. You own everything I produce, even if you stop there.
- 03
Build 路 3 to 6 weeks
I implement the eval suite and CI gates inside your repo, working with your engineers, not around them.
After the audit
The audit shows which of these comes next. Same core every time: evals, tests and release gates.
Model Migration Assurance
Pays for itselfFixed project 路 2 to 3 weeks
Moving to a cheaper model or another provider? I run your golden dataset on both, compare quality, cost and latency per use case and tell you where it's safe to switch and where it isn't.
AI Red Teaming + Evidence Pack
Fixed project 路 paid quarterly retest
Prompt injection, jailbreaks, PII leaks, agent tool misuse. You get a test evidence report ready for security questionnaires, SOC 2, HIPAA and enterprise procurement. QA evidence, not a pentest.
Fractional AI Quality Lead
Monthly 路 defined deliverables
Your part-time head of AI quality until you hire one. I review prompt and model changes, maintain datasets, set metrics and train your team.
Start with the audit
Voice Agent Reliability Audit
Two weeks. Fixed price. A clear answer to "where does our agent fail on real calls, and what do we fix first?"
From $4,500
- Failure mode map of up to 3 call flows
- Starter scenario suite built from your transcripts
- Baseline scores for task completion, tool calls and escalation
- Prioritized risk report and fix plan
- 60-min walkthrough with your team
Read the code first
llm-eval-guardrails
The core harness is public: semantic, schema and groundedness assertions, regression snapshots and a CI gate that blocks the deploy. Voice scenario suites are built on top of it.
FAQ
Stop guessing whether your AI works.
30 minutes, one AI feature, an honest assessment.
Book a reliability review