Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scale

About

Affordance grounding aims to localize where to interact with an object, a fundamental capability for embodied agents. Yet progress is bottlenecked by data: manual annotation is prohibitively expensive and confines existing datasets to a narrow set of predefined object and affordance categories. We introduce Affogato, a framework for open-vocabulary affordance grounding centered on Affogato-750K, a large-scale dataset of 750K 3D affordance heatmaps paired with natural language queries. We build it with a fully automated pipeline that orchestrates foundation models to generate them at scale without human labeling. It covers significantly more diverse categories than any existing dataset. For reliable evaluation, we further provide 5K human-verified test pairs. We also present Espresso-3D and Espresso-2D, simple yet effective models with a unified architecture across both modalities. Pretraining on Affogato-750K improves both Espresso and prior methods and yields the largest gains on unseen object and affordance categories, showing that it provides broadly transferable supervision across architectures.

Junha Lee, Eunha Park, Chunghyun Park, Dahyun Kang, Minsu Cho• 2025

Related benchmarks

TaskDatasetResultRank
Affordance GroundingAGD20k v1 (Seen)
KLD1.426
28
Affordance GroundingAGD20k v1 (Unseen)
KLD1.571
28
Affordance predictionAGD20K unseen
KLD1.034
26
2D affordance groundingAGD20K unseen (test)
KLD0.974
14
3D Affordance GroundingAffogato (test)
aIoU13.6
14
Open-vocabulary 3D affordance groundingLASO seen (test)
aIoU21.9
10
Open-vocabulary 3D affordance groundingLASO unseen (test)
aIoU20.8
10
3D Affordance GroundingAffogato-750K All -> All
aIoU13.6
3
3D Affordance GroundingAffogato-750K Daily-used -> Furnitures
aIoU (Affogato-750K Furnitures)18.2
3
3D Affordance GroundingAffogato-750K Furnitures -> Daily-used
aIoU4.6
3
Showing 10 of 10 rows

Other info

Follow for update