The demo answered in a blink. On a real phone line, your caller says “Hello? Are you still there?”
Short answer: AI voice agents are three models taking turns on a phone line: speech-to-text, a language model and text-to-speech. Budget $0.08 to $0.19 a minute and a pause near 1.5 seconds. Pick Vapi for control, Retell for a faster build, ElevenLabs for the voice.
We built a small one on our own server and timed every stage.
The slowest part wasn’t a model. It was a timer.
How we checked this (and what we didn’t)
On 29 October 2026 we read the live pricing pages of Vapi, Retell, ElevenLabs, Bland and Synthflow. Then we built a tiny voice pipeline on an 8-core cloud server with no GPU.
It ran free parts: Whisper, two small Qwen2.5 models in Ollama, and the Piper voice. Five caller lines from a made-up dental office went in, as clean audio and as 8 kHz phone audio.
We didn’t place live calls on Vapi, Retell or ElevenLabs for this post. Real phone-line timings come from Openbenchmarks, an independent test that dialled five platforms with a robot caller over about 2,000 turns.
Disclosure: we earn a commission if you sign up for Vapi, Retell or ElevenLabs through our links. It doesn’t change what you pay.
What happens in the second and a half after a caller stops talking?
Every turn runs five steps. The phone line carries the audio and speech-to-text writes it down. A turn detector decides the caller is done, the model replies, and text-to-speech reads it out.
| Step | Its job | What goes wrong |
|---|---|---|
| Phone line (Twilio, Telnyx, SIP) | Carries 8 kHz audio both ways | Network delay you never see in a browser demo |
| Speech-to-text (STT) | Turns speech into words as it arrives | Mishears names, numbers and accents |
| Turn detection (endpointing) | Decides the caller has finished | Waits too long, or cuts people off mid-number |
| Language model (LLM) | Writes the reply and calls tools | Slow answers, wrong answers, ignored rules |
| Text-to-speech (TTS) | Speaks the reply | A robotic voice or a slow first word |
The trick is overlap. Good platforms transcribe while the caller is still talking, and start speaking sentence one before the model has written sentence two.
So which step decides how fast your agent feels? Not the one most people blame.
We built a tiny one on our own server. Here’s where the time went
Our fastest stack spent 0.6 seconds thinking per turn; the most accurate, 1.16 seconds. Medians over three runs of five caller lines:
| Stage | What we ran | Median time | What we saw |
|---|---|---|---|
| Speech-to-text | Whisper tiny.en | 0.23 s | Wrote “Del to dental” for Delta Dental, and “tease” for teeth on phone audio |
| Speech-to-text | Whisper base.en | 0.36 s | All five lines right in both audio qualities |
| Speech-to-text | Whisper small.en | 0.86 s | Slower, and still wrote “delto dental” once |
| Language model | Qwen2.5 1.5B | 0.22 s to first sentence | Fast, but ignored the transfer rule |
| Language model | Qwen2.5 3B | 0.65 s to first sentence | Talked about transferring. Didn’t do it |
| Text-to-speech | Piper (lessac voice) | 0.15 s to first sentence | Flat voice, quick start |
Names break first. Every transcription error hit a name or a key word. Give the transcriber your business name, street and insurers before day one. ElevenLabs calls this keyterm prompting.
Bigger models make callers wait. The 3B model tripled the wait for the first spoken sentence. That’s the gap between a snappy reply and a caller repeating themselves.
Saying “I’ll transfer you” isn’t transferring. We told both models to hand anyone who asked over to a person. The 1.5B model said, “Yes, you can speak to a real person,” and did nothing.
Real platforms move the call through a tool call. That’s the line between an agent and a chatbot, and our guide to AI agents vs traditional automation draws it clearly.
Our take: test the boring paths first. Transfers, bookings and “can I speak to someone” break long before small talk does.
Why the demo feels instant and your phone line doesn’t
Vendor latency numbers stop at the vendor’s server. Your caller also hears the turn-detection wait and the phone network.
Retell’s docs say its platform gets as low as 600 milliseconds, measured on its own servers. Openbenchmarks timed the gap from the caller’s side, off real call recordings:
| Platform | Median pause | Slowest 1 in 20 turns |
|---|---|---|
| Telnyx | 1.30 s | 1.86 s |
| ElevenLabs | 1.42 s | 1.77 s |
| Bland | 1.52 s | 2.25 s |
| Vapi | 1.56 s | 2.01 s |
| Retell | 1.74 s | 2.26 s |
None got under one second. Each platform’s own report ran about half a second faster than the recordings.
Now the timer. Vapi waits 0.4 seconds after the caller stops by default, and 1.5 seconds when the transcript lacks punctuation. Retell shipped with a full second.
The benchmark cut both to 0.1 seconds for a fair race, so an out-of-the-box agent pauses longer than the table shows.
Add Vapi’s default wait to our lab stacks and you’re at 1.0 to 1.6 seconds before audio even reaches a phone line. People in normal conversation leave about 200 milliseconds, per a 2009 study of 10 languages.
Why does that matter? Past about two seconds of silence, callers assume the line dropped and start talking over the reply.
Real-world tip: shorten the wait for quick questions and lengthen it where callers read out numbers. A 0.2-second wait will cut someone off halfway through a phone number.
What does one minute really cost on Vapi, Retell, ElevenLabs and Bland?
For a typical small-business agent, budget $0.08 to $0.19 per minute. The platform fee is the line everyone sees. It’s rarely the biggest.
- Vapi charges $0.05 a minute for hosting and passes the parts through at cost. That’s about $0.01 for Deepgram transcription, $0.008 to $0.045 for an OpenAI model and $0.015 to $0.024 for an ElevenLabs voice. Calls on a Vapi number carry no transport fee.
- Retell charges $0.055 for its infrastructure. On top go the model (under a cent to $0.16), the voice ($0.015 for its own, $0.04 for ElevenLabs Flash) and $0.015 for a US Twilio line.
- ElevenLabs Agents charge $0.08 a minute for the voice side. The model is billed on top, and your carrier bills you directly.
- Bland charges $0.14 a minute with the model, transcription and voice included, or $0.12 on a $299 monthly plan. Telephony is extra.
Synthflow has no self-serve price; contracts start at $30,000 a year.
Here’s the maths for a dental office taking 1,000 US calls a month at three minutes each: 3,000 minutes.
| Setup | Per minute | Per month | Not included |
|---|---|---|---|
| Vapi, cheap to premium parts | $0.082 to $0.129 | $245 to $387 | Twilio if you bring a number (+$24) |
| Retell, GPT 5.4 mini + Retell voice | $0.109 | $327 | Knowledge base (+$15) |
| Retell, GPT 5.4 + ElevenLabs Flash | $0.190 | $570 | Knowledge base (+$15) |
| ElevenLabs Pro + extra minutes | $0.08 + model | $240 + model | Model, carrier (~$24 on Twilio) |
| Bland Start | $0.14 | $420 | Telephony |
The benchmark’s own invoices agree. On default settings, a minute cost $0.079 on ElevenLabs, $0.084 on Vapi, $0.134 on Retell and $0.141 on Bland, before the carrier.
One quirk on ElevenLabs: every paid plan works out to exactly $0.08 a minute. $22 buys 275 minutes and $99 buys 1,238, so you’re really buying how many calls can run at once. Our ElevenLabs pricing breakdown explains the plans behind it.
And Bland’s $299 plan only pays off above 14,950 minutes a month. Below that, $0.14 is cheaper.
The model you pick moves the bill more than the platform
On Retell’s price list, the model line runs from $0.0064 a minute for GPT 5.4 nano to $0.16 for GPT 5.5. Over our 3,000 minutes, that’s $19 versus $480 for the same calls.
Speech-to-speech models skip the relay: one model hears and speaks. They sound more natural and cost more. GPT Realtime is $0.345 a minute on Retell, or $1,035 a month for our dental office. The mini version is $0.07.
Wait. Didn’t the small model fail our transfer test?
It did, and that’s the trade. A mini-class model handles hours, prices and bookings for pennies. Test every action it must take, and move up a tier only when a real call fails.
Answering from your own documents uses the same retrieval trick as chatbots. Our plain-English RAG explainer shows why short, clean pages make better phone answers.
The extras that show up after launch
Concurrency is the bill nobody budgets for. It’s how many calls can run at the same moment.
| Extra | Vapi | Retell | ElevenLabs | Bland |
|---|---|---|---|---|
| Free credit | $5 | $10 | 15 minutes | 2 credits |
| Calls at once (entry) | 4 | 20 | 4 free, 20 on Pro | 10 |
| More calls at once | $10 per line a month | $8 per call a month | Burst at $0.16 a minute | 50 on the $299 plan |
| Phone number | 1 included | $2 a month | Bring your own | Inbound number on Start |
Picture that dental office at 9 a.m. on a Monday. Five people call at once, and Vapi’s free tier has four lines. The fifth caller needs a $10 monthly line, or a busy signal.
Retell’s add-ons stack up too: knowledge base, noise removal and guardrails add half a cent a minute each, and PII removal a full cent.
Vapi, Retell, ElevenLabs or Bland: which one fits you?
| Feature |
Bland$0.14/minBest for: One all-in rate with no token maths |
|||
|---|---|---|---|---|
| What the rate covers | Hosting only | Infrastructure only | Voice side, not the model | Model, transcription and voice |
| Calls at once, free | 4 | 20 | 4 (Free plan) | 10 |
| Median pause on real calls | 1.56 s | 1.74 s | 1.42 s | 1.52 s |
| Our 3,000-minute month | $245 to $387 | $327 to $570 | $240 + model | $420 + telephony |
- What the rate covers
- Hosting only
- Calls at once, free
- 4
- Median pause on real calls
- 1.56 s
- Our 3,000-minute month
- $245 to $387
- What the rate covers
- Infrastructure only
- Calls at once, free
- 20
- Median pause on real calls
- 1.74 s
- Our 3,000-minute month
- $327 to $570
- What the rate covers
- Voice side, not the model
- Calls at once, free
- 4 (Free plan)
- Median pause on real calls
- 1.42 s
- Our 3,000-minute month
- $240 + model
- What the rate covers
- Model, transcription and voice
- Calls at once, free
- 10
- Median pause on real calls
- 1.52 s
- Our 3,000-minute month
- $420 + telephony
We may earn a commission when you buy through links on this page. This does not affect our reviews.
Pick Vapi if you write code and want to choose every part, because it charges a flat hosting fee and passes the rest through at cost. Skip it if you need more than four calls at once and don’t want to pay per line. Our full Vapi review walks through the builder.
Pick Retell if you’d rather start from a template with 20 free simultaneous calls, because a busy front desk hits concurrency limits first. Skip it if you want premium voices cheaply, since ElevenLabs voices add $0.04 to $0.10 a minute there. More in our Retell review.
Pick ElevenLabs if callers will judge the voice first, or you want to clone your own voice for the agent. Skip it if you want phone numbers handled for you, because you bring your own carrier.
Pick Bland if you hate per-token maths and want one number per minute. Skip it if volume is low and every cent counts: $0.14 sits above our Vapi and ElevenLabs setups.
Writing bookings into a CRM? Retell lists n8n as an integration, and our n8n review covers that glue layer.
Frequently asked questions
Can an AI voice agent use my existing business phone number?
Yes. The simplest route is forwarding your current number to the agent’s number. Bigger setups connect a SIP trunk from their carrier: Retell charges nothing for custom telephony, Vapi supports Twilio, Vonage and Telnyx, and ElevenLabs connects to Twilio or SIP. Porting the number itself is possible but slower.
Is it legal to make outbound calls with an AI voice agent?
It depends on where you call. In the US, the FCC ruled in February 2024 that AI-generated voices count as artificial voices under the TCPA, so those calls need the called person’s prior express consent. Answering inbound calls is far simpler. Check your country’s rules before any outbound campaign.
What happens when the voice agent can’t answer a question?
You decide in advance. Most setups transfer to a person, take a message, or text a link. Retell offers call transfer and SMS at $0.01 a message, and Bland bills transferred minutes at $0.04 to $0.05. Write the fallback into the prompt and test it, because a dead end loses the caller.
What is a good latency for an AI voice agent?
Retell recommends keeping an agent’s estimated latency under 1.5 seconds and treats a median above 2.5 seconds on real calls as high. Watch the slowest turns, not the average: Retell’s own example call had a 0.8-second median but a worst turn of 2.5 seconds.
Can a voice agent book appointments into my calendar?
Yes, through tool calls. The model decides a booking is needed, the platform calls your calendar or a webhook, and the agent reads back the slot. Retell lists appointment booking as a built-in feature. On Vapi you wire it up with functions or an automation tool.
Do AI voice agents work in languages other than English?
Yes, with caveats. ElevenLabs lists 70 or more languages for its conversational voices. Turn detection is the weak spot: Vapi recommends LiveKit endpointing for English and its own endpointing for other languages. Test with native speakers, because accents hit transcription hardest.
Can I self-host an AI voice agent to avoid per-minute fees?
You can. Frameworks like LiveKit Agents and Pipecat let you run your own stack, and our CPU test ran open-source parts for free. You then own the phone connection, scaling and uptime, and good models need a GPU. Below a few thousand minutes a month, a platform usually costs less than your time.
How do I test a voice agent before customers call it?
Call it yourself from a mobile on a weak signal, then try to break it: interrupt it, read out a phone number slowly, ask for a human. Vapi and Retell both include call simulation, and Retell’s call logs show latency percentiles for every turn, so slow tails are easy to spot.
The bottom line
AI voice agents live or die on the pause, and that pause comes mostly from timers, model size and the network.
Developers should start on Vapi. Teams that want templates and 20 free lines should start on Retell. Brands that sell on their voice should start on ElevenLabs Agents.
One warning: budget for the model and for concurrency, not the platform fee.
Your next step: build the agent, then call it from your own phone ten times. Count the pauses before you count the features.
