ProactiveAudioBench: Evaluating Condition-Driven Responses in Audio Streams

Abstract

Audio assistants are moving from turn-based question answering toward continuous listening, where they must interpret incoming audio while deciding whether to respond or remain silent. Existing proactive benchmarks emphasize video and audiovisual interaction, while audio-centric evaluations offer limited systematic coverage of listening conditions across audio domains. We introduce ProactiveAudioBench, a benchmark for condition-driven responses across sound events, speech, and music. Four tasks—Detection, Tracking, Counting, and Matching—cover first-occurrence, repeated, count-conditioned, and reference-based notifications. Natural-language conditions and optional acoustic references are paired with annotated response opportunities and reference replies, supporting both single-response and multiple-response requests. The framework accommodates simulated and native streaming and defines joint time-and-content Coverage alongside negative-request False Alarm Rate. An initial offline study evaluates seven open-source audio models across 56 subsets, yielding 27,120 prediction records. Using first-trigger diagnostics, the highest reported timing F1 at a ±3s tolerance is 0.292. Baichuan-Omni-1.5 and Qwen Thinking have negative-request false-trigger rates of 0.581 and 0.861 on their respective request sets, whereas the other five models have response-presence F1 no greater than 0.113. These subset-level results reveal difficulty balancing timely notifications with appropriate silence. ProactiveAudioBench provides a testbed for connecting audio understanding with reliable execution of persistent listening instructions.