Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

InterPartAbility: Text-Guided Part Matching for Interpretable Person Re-Identification

About

Text-to-image person re-identification (TI-ReID) relies on natural-language text description to retrieve top matching individuals from a large gallery of images. While recent large vision-language models (VLMs) achieve strong retrieval performance, their decisions remain largely uninterpretable. Existing interpretability approaches in TI-ReID rely solely on slot-attention to highlight attended regions, but fail to reliably bind visual regions to semantically meaningful concepts, limiting explanations to qualitative visualizations over a restricted vocabulary. This paper introduces InterPartAbility, an interpretable TI-ReID method that performs explicit part-wise matching and enables phrase-region grounding. A new open-vocabulary, lightweight supervision, patch-phrase interaction module (PPIM) is proposed to train a standard TI-ReID model with concept-level guidance. Concept-based part phrases provide evidence that encourages the model to attend to corresponding image regions. InterPartAbility further constrains CLIP ViT self-attention to produce spatially concentrated patch activations aligned with each part-level phrase, yielding grounded explanation maps. A quantitative interpretability protocol for TI-ReID is introduced by adapting perturbation-based evaluation metrics, including counterfactual region masking that measures retrieval degradation when top-ranked explanatory regions are removed. Empirical results\footnote{Our code is included in the supplementary materials and will be made public.} on challenging benchmarks like CUHK-PEDES and ICFG-PEDES show that InterPartAbility achieves state-of-the-art (SOTA) interpretability performance under these metrics, while sustaining competitive retrieval accuracy.

Shakeeb Murtaza, Aryan Shukla, Rajarshi Bhattacharya, Maguelonne Heritier, Eric Granger• 2026

Related benchmarks

TaskDatasetResultRank
Text-to-Image RetrievalCUHK-PEDES (test)
Recall@178.17
114
Text-to-image person retrievalRSTPReid
Rank-1 Accuracy70.9
66
Text-based Person Re-identificationRSTPReid
Rank-1 Accuracy70.9
57
Text-to-image Person Re-identificationCUHK-PEDES
Rank-178.17
51
Text-based Person Re-identificationICFG-PEDES
R@169.92
36
Text-to-Image RetrievalICFG-PEDES
R@169.92
8
Text-to-Image RetrievalRSTPReid (test)
Delta R@1%10.87
3
Showing 7 of 7 rows

Other info

Follow for update