Few to Big: Prototype Expansion Network via Diffusion Learner for Point Cloud Few-shot Semantic Segmentation
About
Few-shot 3D point cloud semantic segmentation aims to segment novel categories using a minimal number of annotated support samples. However, prototypes derived from the limited non-structural point cloud support set are often misaligned and have a small capacity, hindering effective gen eralization to novel categories. This stems from two core issues: i) the prototype possess limited representational capacity fails to cover the full intra-class diversity of a novel category, and ii) the prototypes suffer from misalignment with the query space due to the inter-set inconsistency between support and query sets. To address these issues, our work focuses on leveraging the few support samples to construct a well-aligned big-capacity prototype. Motivated by the powerful generative capabilities of diffusion models, we re-purpose its pre-trained conditional encoder to provide rich feature components for prototype ex pansion. Subsequently, a push-pull force aligns this expanded prototype towards the query feature space. Under this setup, we introduce the Prototype Expansion Network (PENet), a framework that constructs aligned big-capacity prototypes from two complementary feature sources. Specifically, PENet employs a dual-stream learner architecture: it retains a conventional fully supervised Intrinsic Learner (IL) to distill representative features, while introducing a novel Diffusion Learner (DL) to provide rich generalizable features. The resulting dual prototypes are then processed by a Prototype Assimilation Module (PAM), which adopts a push-pull attention block to align the prototypes with the query space. Furthermore, a Prototype Calibration Mechanism (PCM) regularizes the final big-capacity prototype to prevent semantic drift. Extensive experiments on the S3DIS and ScanNet datasets demonstrate that PENet outperforms state-of-the-art methods across various few-shot settings.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| 3D Semantic Segmentation | ScanNet S0 | mIoU73.15 | 80 | |
| 3D Point Cloud Semantic Segmentation | ScanNet official (fold S1) | mIoU71.51 | 68 | |
| Few-shot 3D Semantic Segmentation | S3DIS (S1) | mIoU79.37 | 67 | |
| Few-shot 3D Semantic Segmentation | S3DIS (S0) | mIoU72.33 | 64 | |
| 3D Few-shot Semantic Segmentation | S3DIS Average | mIoU (%)75.85 | 44 | |
| 3D Point Cloud Semantic Segmentation | ScanNet Average of S0 and S1 | mIoU72.33 | 44 | |
| Few-shot 3D Semantic Segmentation | ScanNet | mIoU68.86 | 23 | |
| 3D Point Cloud Semantic Segmentation | SemanticPOSS | S0 Score53.3 | 12 |