Streaming Speech Perception

Replacing the VAD+ASR pipeline with a streaming perception model: continuous audio stream instead of chopped segments, full context for in-context learning, higher accuracy for names and terms. The model outputs not only text but also tokens for acoustic events: speech start/end, interruption, emotion, laughter, noise – a single stream of events for the reasoning layer. Turn-taking detection can be built directly into the recognizer (semantics + silence instead of a silence threshold); key finding: fluctuations in “whether to take a turn” are often cured by re-labeling training labels “with only information available at the decision moment”.

Related: [Cascaded Voice Pipeline], [Voice Paradigms]