Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence

About

Establishing dense correspondence across 3D shapes is crucial for fundamental downstream tasks, including texture transfer, shape interpolation, and robotic manipulation. However, learning these mappings without manual supervision remains a formidable challenge, particularly under severe non-isometric deformations and in inter-class settings where geometric cues are ambiguous. Conventional functional map methods, while elegant, typically struggle in these regimes due to their reliance on isometry. To address this, we present GLASS, a framework that bridges the gap by integrating geometric spectral analysis with rich semantic priors from vision-language foundation models. GLASS introduces three key innovations: (i) a view-consistent strategy that enables robust multi-view visual feature extraction from powerful vision foundation models; (ii) the injection of language embeddings into vertex descriptors via zero-shot 3D segmentation, capturing high-level part semantics; and (iii) a graph-assisted contrastive loss that enforces structural consistency between regions (e.g., source's head'' $\leftrightarrow$ target's head'') by leveraging geodesic and topological relationships between regions. This design allows GLASS to learn globally coherent and semantically consistent maps without ground-truth supervision. Extensive experiments demonstrate that GLASS achieves state-of-the-art performance across all regimes, maintaining high accuracy on standard near-isometric tasks while significantly advancing performance in challenging settings. Specifically, it achieves average geodesic errors of 0.21, 4.5, and 5.6 on the inter-class benchmark SNIS and non-isometric benchmarks SMAL and TOPKIDS, reducing errors from URSSM baselines of 0.49, 6.0, and 8.9 by 57%, 25%, and 37%, respectively.

Qinfeng Xiao, Guofeng Mei, Qilong Liu, Chenyuan Yi, Fabio Poiesi, Jian Zhang, Bo Yang, Yick Kit-lun• 2026

Related benchmarks

TaskDatasetResultRank
Non-isometric 3D shape matchingSMAL
Mean Geodesic Error0.045
58
Shape correspondence estimationTOPKIDS
Geodesic Error (x100)5.6
44
Inter-class shape matchingSNIS (test)
Average Geodesic Error0.21
14
3D shape matchingSCAPE remeshed
Average Geodesic Error (x100)1.9
9
3D shape matchingSHREC19 remeshed
Average Geodesic Error3.1
9
3D shape matchingFAUST remeshed
Average Geodesic Error0.016
9
Showing 6 of 6 rows

Other info

Follow for update