Cascaded Voice Pipeline

Pipeline:

VAD → ASR → LLM → TTS

VAD determines the end of speech (silence threshold of 500–800ms – a trade-off: shorter cuts off thinking pauses, longer – an extra second of waiting).

Each stage waits for the previous one: a total of ~0.95–2.3 seconds even without load; the delay in the queue grows non-linearly (~S/(1-load)). While VAD and ASR are working, the LLM is idle.

Streaming processing hides some of the latency: ASR transcribes during speech, the LLM segments the response and sends the first fragment to TTS without waiting for the entire response, TTS synthesizes incrementally. But determining the end of the utterance remains.