CoCo-SAM3: Harnessing Concept Conflict in Open-Vocabulary Semantic Segmentation
About
SAM3 advances open-vocabulary semantic segmentation by introducing a prompt-driven mask generation paradigm. However, in multi-class open-vocabulary scenarios, masks generated independently from different category prompts lack a unified and inter-class comparable evidence scale, often resulting in overlapping coverage and unstable competition. Moreover, synonymous expressions of the same concept tend to activate inconsistent semantic and spatial evidence, leading to intra-class drift that exacerbates inter-class conflicts and compromises overall inference stability. To address these issues, we propose CoCo-SAM3 (Concept-Conflict SAM3), which explicitly decouples inference into intra-class enhancement and inter-class competition. Our method first aligns and aggregates evidence from synonymous prompts to strengthen concept consistency. It then performs inter-class competition on a unified comparable scale, enabling direct pixel-wise comparisons among all candidate classes. This mechanism stabilizes multi-class inference and effectively mitigates inter-class conflicts. Without requiring any additional training, CoCo-SAM3 achieves consistent improvements across eight open-vocabulary semantic segmentation benchmarks.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Semantic segmentation | ADE20K | mIoU38.3 | 567 | |
| Semantic segmentation | Cityscapes | mIoU70.7 | 497 | |
| Semantic segmentation | PASCAL VOC with background category VOC21 2012 | mIoU86.8 | 51 | |
| Semantic segmentation | Average Overall | mIoU64.3 | 46 | |
| Semantic segmentation | Pascal Context 60 with background | mIoU50.5 | 43 | |
| Semantic segmentation | Pascal VOC without background 2012 V20 | mIoU95.2 | 42 | |
| Semantic segmentation | COCO-Object with background class | mIoU67.9 | 34 | |
| Semantic segmentation | Pascal Context 59 (PC59) without background | mIoU61.2 | 20 | |
| Semantic segmentation | COCO-S without background | mIoU43.6 | 19 |