Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Solution for 10th Competition on Ambivalence/Hesitancy (AH) Video Recognition Challenge using Divergence-Based Multimodal Fusion

About

We address the Ambivalence/Hesitancy (A/H) Video Recognition Challenge at the 10th ABAW Competition (CVPR 2026). We propose a divergence-based multimodal fusion that explicitly measures cross-modal conflict between visual, audio, and textual channels. Visual features are encoded as Action Units (AUs) extracted via Py-Feat, audio via Wav2Vec 2.0, and text via BERT. Each modality is processed by a BiLSTM with attention pooling and projected into a shared embedding space. The fusion module computes pairwise absolute differences between modality embeddings, directly capturing the incongruence that characterizes A/H. On the BAH dataset, our approach achieves a Macro F1 of 0.6808 on the validation test set, outperforming the challenge baseline of 0.2827. Statistical analysis across 1{,}132 videos confirms that temporal variability of AUs is the dominant visual discriminator of A/H.

Aislan Gabriel O. Souza, Agostinho Freire, Leandro Honorato Silva, Igor Lucas B. da Silva, Jo\~ao Vin\'icius R. de Andrade, Gabriel C. de Albuquerque, Lucas Matheus da S. Oliveira, M\'ario Stela Guerra, Luciana Machado• 2026

Related benchmarks

TaskDatasetResultRank
Video-level A/H RecognitionBAH (dev val)
MF169.12
31
Video-level A/H RecognitionBAH (test)--
28
Showing 2 of 2 rows

Other info

Follow for update