A Baseline Study and Benchmark for Few-Shot Open-Set Action Recognition with Feature Residual Discrimination
About
Few-Shot Action Recognition (FS-AR) has shown promising results but is often limited by a closed-set assumption that fails in real-world open-set scenarios. While Few-Shot Open-Set (FSOS) recognition is well-established for images, its extension to spatio-temporal video data remains underexplored. To address this, we propose an architectural extension based on a Feature-Residual Discriminator (FR-Disc), adapting previous work on skeletal data to the more complex video domain. Extensive experiments on five datasets demonstrate that while common open-set techniques provide only marginal gains, our FR-Disc significantly enhances unknown rejection capabilities without compromising closed-set accuracy, setting a new state-of-the-art for FSOS-AR. The project website, code, and benchmark are available at: https://hsp-iit.github.io/fsosar/.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Few-Shot Open-Set Action Recognition | Diving48 | FS ACC78.58 | 12 | |
| Few-Shot Open-Set Action Recognition | SS v2 | FS Acc77.88 | 8 | |
| Few-Shot Open-Set Action Recognition | NTURGBD | FS Accuracy95.54 | 8 | |
| Few-Shot Open-Set Action Recognition | HMDB51 | FS Accuracy85.17 | 8 | |
| Few-Shot Open-Set Action Recognition | UCF101 | FS Accuracy99.28 | 8 | |
| Open set action recognition | SSv2 (test) | FS ACC65.51 | 4 | |
| Open set action recognition | UCF101 (test) | FS Accuracy95.76 | 4 | |
| Open set action recognition | NTURGBD (test) | FS ACC93.28 | 4 | |
| Open set action recognition | HMDB51 (test) | FS Accuracy75.5 | 4 |