Time to first token (TTFT) is the metric teams use to pick an inference API for voice. It is also the metric that misleads them. TTFT marks when generation starts; a text-to-speech model cannot speak until a full clause arrives. Between those two points sits the difference between an agent that feels conversational and one that gets interrupted. This piece benchmarks every layer of the voice stack including LLM, speech-to-text, text-to-speech, and speech-to-speech.
Why TTFT Is the Right Entry Point and the Wrong Finish Line
A voice agent is a latency budget with a language model inside it. Every stage spends milliseconds the user can hear.
Time to first token (TTFT) is the interval between sending an inference request and receiving the first token back. IBM’s definition frames it as the moment a system transitions from idle to visibly active.
For chat, TTFT is close to the whole story. For voice, it is one term in a sum.
The reason is mechanical. A text-to-speech model cannot synthesise half a word. It needs a complete clause or sentence before it produces audio. LiveKit calls the resulting metric time-to-first-sentence (TTFS), and argues in its Gemma 4 deployment post that TTFS is what users actually feel.
That gives you two knobs rather than one. TTFT controls when generation starts. Tokens per second controls how fast the first sentence completes. A provider that wins one and loses the other will not feel fast.
The Latency Budget: What One Voice Turn Actually Costs
LiveKit’s voice agents overview breaks a turn into STT at roughly 100–200ms, LLM at 300–500ms with streaming, TTS at 100–200ms, and network at 50–150ms over WebRTC. It puts the practical end-to-end target at 700ms to 1.2s.
Kwindla Hultman Kramer, co-creator of Pipecat, has advised targeting 800ms median voice-to-voice latency, with a looser 1,500ms acceptable for a proof of concept. His rough arithmetic splits that four ways at roughly 200ms each: transport and media processing, STT plus phrase endpointing, LLM inference, and TTS.
Daily’s earlier work on the fastest voice bot supplies the human baseline. Typical human response time in conversation is around 500ms. Pauses beyond 800ms start to feel unnatural.
Daily’s February 2026 voice-agent LLM benchmark translates that into an LLM requirement directly. Natural conversation needs voice-to-voice under 1,500ms, which works out to roughly 700ms of TTFT budget for a text-mode LLM inside a transcription-to-LLM-to-voice harness.
That 700ms number is the bar to hold every provider against.
How to Read a TTFT Benchmark Without Being Misled
Before the tables, five methodology facts that change what the numbers mean:
1. Workload shape dominates: Artificial Analysis changed its default workload in March 2026. The site now reports 10k input token prompts rather than 1k. Longer prompts raise both TTFT and output speed. LiveKit argues this is closer to reality for voice, because production agents front-load policy, persona, escalation rules, retrieved data, and tool schemas.
2. Server location is baked in: Artificial Analysis tests from a virtual machine in Google Cloud’s us-central1-a zone. It states plainly that TTFT includes network latency and may advantage or disadvantage providers based on where they serve.
3. Reasoning tokens count: In the Artificial Analysis definition, TTFT for a reasoning model is the first reasoning token, not the first answer token. Those are separate columns.
4. Measure from the receiving side: Daily notes that model providers sometimes quote TTFT internal to their inference stacks. Daily measures from request send to first usable token off the API.
5. Runs are not repeatable: Daily is blunt about this: TTFT varies substantially between benchmark runs, and providers change inference stacks and sometimes weights without changing model names.
Layer 1: LLM Time to First Token
Figures below are from the Artificial Analysis API providers leaderboard, retrieved August 30, 2026. The “first chunk” column is TTFT. Workload is 10k input tokens, single prompt, median over 72 hours.
Lowest measured first-chunk latency
| Provider | Model | TTFT | Output speed |
|---|---|---|---|
| Baseten | gpt-oss-120b (high) | 0.23s | 266 tok/s |
| Baseten | gpt-oss-120b (low) | 0.24s | 271 tok/s |
| DeepInfra | Nemotron 3 Ultra | 0.28s | 371 tok/s |
| Cohere | North Mini Code | 0.32s | 104 tok/s |
| Cohere | Command A+ | 0.40s | 239 tok/s |
| Baseten | Inkling Small | 0.42s | 337 tok/s |
| Modular | Gemma 4 31B (NVFP4) | 0.44s | 243 tok/s |
| Nebius | GLM-5.3-Flash | 0.46s | 206 tok/s |
| Fireworks | Nemotron 3.5 Lightning | 0.46s | 501 tok/s |
| Together AI | Kimi K2.7 Code | 0.47s | 245 tok/s |
| Cerebras | gpt-oss-120b (high) | 0.49s | 1,697 tok/s |
The throughput trap
Silicon vendors optimise for a different metric than voice agents need.
| Provider | Model | TTFT | Output speed |
|---|---|---|---|
| Cerebras | gpt-oss-120b (high) | 0.49s | 1,697 tok/s |
| Celeris | Celeris-1 | 0.62s | 1,612 tok/s |
| Cerebras | Gemma 4 31B | 0.53s | 1,351 tok/s |
| Groq | gpt-oss-20b (high) | 0.82s | 957 tok/s |
| SambaNova | gpt-oss-120b (high) | 0.92s | 706 tok/s |
| Groq | gpt-oss-120b (low) | 0.69s | 473 tok/s |
| Inception | Mercury 2 | 3.07s | 770 tok/s |
Mercury 2 is the clearest illustration. It is a diffusion-based language model, and it generates 770 tokens per second. Its first chunk arrives at 3.07s. That is four times the entire LLM budget for a natural conversation.
Cerebras and Groq are a different case. Their TTFT is respectable and their throughput is exceptional. For TTFS specifically, that combination is strong, because the sentence completes almost immediately after the first token lands.
Frontier and proprietary endpoints
| Provider | Model | TTFT | Output speed |
|---|---|---|---|
| Amazon Bedrock | GPT-5.6 Luna (non-reasoning) | 0.59s | 181 tok/s |
| Amazon Bedrock | GPT-5.6 Terra (non-reasoning) | 0.72s | 103 tok/s |
| Open Source Read original → Related articles   |