Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding

About

Recent advances in Streaming Video Understanding has enabled a new interaction paradigm where models respond proactively to user queries. Current proactive VideoLLMs rely on per-frame triggering decision making, which suffers from an efficiency-accuracy dilemma. We propose Em-Garde, a novel framework that decouples semantic understanding from streaming perception. At query time, the Instruction-Guided Proposal Parser transforms user queries into structured, perceptually grounded visual proposals; during streaming, a Lightweight Proposal Matching Module performs efficient embedding-based matching to trigger responses. Experiments on StreamingBench and OVO-Bench demonstrate consistent improvements over prior models in proactive response accuracy and efficiency, validating an effective solution for proactive video understanding under strict computational constraints.

Yikai Zheng, Xin Ding, Yifan Yang, Shiqi Jiang, Hao Wu, Qianxi Zhang, Weijun Wang, Ting Cao, Yunxin Liu• 2026

Related benchmarks

TaskDatasetResultRank
Streaming Video UnderstandingStreamingBench--
158
Online Video UnderstandingOVO-Bench
Backward Tracing Avg.52.2
48
Real-time Visual PerceptionOVO-Bench
OCR76.5
27
Backward TracingOVO-Bench
EPM48.2
27
Proactive Video Question AnsweringProactiveVideoQA EGO
PAUC (ω=0.5)52.3
8
Proactive ResponseStreamingBench
Accuracy38
7
Online Video UnderstandingStreamingBench
Real-time VU Score76.7
6
Proactive Response TimingOVO-Bench Future Active Responding
CRR Recall47.92
5
Proactive Video Question AnsweringProactiveVideoQA WEB (test)
PAUC44.3
4
Proactive Video Question AnsweringProactiveVideoQA VAD (test)
PAUC27.4
4
Showing 10 of 10 rows

Other info

Follow for update