Cascaded Voice Pipeline

VAD determines the end of speech (silence threshold of 500-800ms – a compromise: shorter cuts off thinking pauses, longer – an extra second of waiting) → ASR transcribes → LLM generates → TTS voices. Each stage waits for the previous one: a total of ~0.95-2.3 seconds even without load; the delay in the queue grows non-linearly (~S/(1-load)). While VAD and ASR are working, the LLM is idle. Streaming processing hides some of the latency: ASR transcribes during speech, the LLM segments the response and sends the first fragment to TTS without waiting for the entire response, TTS synthesizes incrementally. But the determination of the end of the utterance remains.

Related: [Voice paradigms], [Streaming speech perception]