Streaming Speech Perception

Replacing the VAD+ASR pipeline with a streaming perception model: continuous audio stream instead of chopped segments, full context for in-context learning, higher accuracy for names and terms.

The model outputs not only text, but also tokens for acoustic events: speech start/end, interruption, emotion, laughter, noise – a single stream of events for the reasoning layer.

Turn-taking detection can be built directly into the recognizer (semantics + silence instead of a silence threshold).

Key finding: hesitations about “whether to take a turn” are often cured by relabeling training labels with “only information available at the decision moment”.