Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Prompt-Calibrated SAM 3 for Open-Vocabulary Remote Sensing Semantic Segmentation

About

Open-vocabulary semantic segmentation (OVSS) in remote sensing images aims to segment categories beyond a fixed label space. Recent SAM 3-based methods provide a promising training-free foundation, yet three key issues remain: (1) a single class-name prompt lacks sufficient semantic coverage for complex remote sensing categories; (2) expanding each category into multiple prompts introduces redundant online text encoding; and (3) directly aggregating multiple prompt responses propagates noisy activations into the final prediction. To address these issues, we propose ProC-SAM3, which calibrates SAM 3's prompt interface for remote sensing OVSS from three complementary aspects. First, we construct an offline prompt pool where a Category Matcher groups MLLM-generated candidates into per-category sets, and Expansion Constraints further refine each set using category-specific prior knowledge. Second, the resulting text embeddings are cached and reused across all test images, eliminating repeated text encoding. Third, we introduce Presence-Guided Residual Fusion to gate unreliable decoder outputs by prompt presence and confidence, followed by peak-preserving class aggregation that retains fine-grained activations for small and sparse objects. Experiments on eight benchmarks show that ProC-SAM3 achieves an average mIoU of 56.1%, outperforming the previous best training-free method by 3.9 percentage points. Code will be available at https://github.com/YanghuiSong/ProC-SAM3.

Yanghui Song, Nanqing Liu, Haonan Yin, Yingjie Gao, Chengfu Yang, Qi Ming• 2026

Related benchmarks

TaskDatasetResultRank
Open-Vocabulary SegmentationVaihingen
mIoU64.5
33
Open-Vocabulary SegmentationLoveDA
mIoU46.1
33
Open-Vocabulary SegmentationPotsdam
mIoU58.8
33
Open-Vocabulary SegmentationiSAID
mIoU29.5
33
Open Vocabulary Semantic SegmentationUAVid
mIoU60
20
Open Vocabulary Semantic SegmentationUDD5
mIoU73.4
20
Open Vocabulary Semantic SegmentationVDD
mIoU0.675
20
Open Vocabulary Semantic SegmentationOEM
mIoU48.9
20
Showing 8 of 8 rows

Other info

Follow for update