Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning

About

Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes from limited examples while preserving knowledge of previously learned classes. Existing methods face a critical dilemma: static architectures rely on a constant parameter space to learn from data that arrive sequentially, making them prone to overfitting to the current session, while dynamic architectures continually expand the parameter space, leading to increased complexity. In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL. Mamba leverages its input-dependent parameters to dynamically adjust its processing patterns and generate content-aware scan patterns without session-wise projector expansion. This enables it to configure distinct processing for base and novel classes, helping preserve existing knowledge while adapting to new ones. To leverage Mamba's potential for FSCIL, we design two key modules: First, we propose a dual selective SSM projector that generates input-conditioned state-space parameters from intermediate features for dynamic adaptation. The dual design structurally decouples base and novel-class processing, employing a frozen base branch to maintain stable base-class features and a dynamic incremental branch that adaptively learns distinctive feature shifts for novel classes. Second, we develop a class-sensitive selective scan mechanism to guide dynamic adaptation of the incremental branch. It reduces the disruption to base-class representations caused by training on novel data, and meanwhile, encourages the selective scan to perform in distinct patterns between base and novel classes. Extensive experiments on miniImageNet, CIFAR-100, and CUB-200 demonstrate that Mamba-FSCIL achieves state-of-the-art performance.

Xiaojie Li, Yibo Yang, Jianlong Wu, Yue Yu, Ming-Hsuan Yang, Liqiang Nie, Min Zhang• 2024

Related benchmarks

TaskDatasetResultRank
Few-Shot Class-Incremental LearningminiImageNet (test)
Accuracy (Session 1)79.23
200
Few-Shot Class-Incremental LearningCUB200 (test)
Accuracy (Session 1)86.96
128
Few-Shot Class-Incremental LearningCIFAR100
Accuracy (S0)82.95
111
Few-Shot Class-Incremental LearningCUB-200
Session 1 Accuracy76.26
85
Few-Shot Class-Incremental LearningCUB-200
Session 0 Accuracy80.92
20
Few-Shot Class-Incremental LearningImageNet mini
Session 8 Harmonic Mean41.72
20
Few-Shot Class-Incremental LearningCUB200
Base Accuracy80.9
17
Few-Shot Class-Incremental LearningCIFAR100
Base Accuracy82.8
15
Showing 8 of 8 rows

Other info

Follow for update