Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning

About

3D affordance grounding aims to understand how diverse objects can be manipulated, making it a cornerstone of embodied interaction. However, prior works struggle to generalize to out-of-distribution, open-world scenarios, leaving a critical gap between limited dataset performance and real-world application needs. Inspired by the saying: \textit{\textbf{``What I can not create, I do not understand''}}, we find generative models can generate semantically valid HOI images, which indicates inherent encoding of affordance concepts. Building on this insight, we propose DAG, the first innovative diffusion-based 3D affordance grounding framework that extracts general affordance knowledge from text-to-image diffusion models for 3D affordance prediction. Specifically, we extract the affordance priors from a diffusion model to encode HOI priors, and design an affordance block with a multi-source affordance decoder for dense 3D affordance prediction. Extensive experiments show that DAG consistently outperforms state-of-the-art methods and exhibits strong open-world generalization, even in the challenging one-shot setting. The code of our method is released on \textcolor{blue}{\textit{https://github.com/hq-King/DAG}}.

Hanqing Wang, Zhenhao Zhang, Kaiyang Ji, Mingyu Liu, Wenti Yin, yuchao chen, Zhirui Liu, Xiangyu Zeng, Tianxiang Gui, Hangxing Zhang, Jiahao Yuan, Zhiqing Cui, Jiaxin Liu, Zhiyuan Ma, Hui Xiong• 2025

Related benchmarks

TaskDatasetResultRank
3D Affordance GroundingPIAD (Seen)
aIOU24.84
20
3D Affordance GroundingPIAD (Unseen)
aIOU9.73
20
3D Affordance GroundingPIAD v2 (Seen)
mIoU47.19
13
3D Affordance GroundingPIAD Unseen Object v2
AUC85.41
13
Affordance GroundingPIAD2 Unseen Aff
mIoU16.09
6
Showing 5 of 5 rows

Other info

Follow for update