Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Open-World Amodal Appearance Completion

About

Understanding and reconstructing occluded objects is a challenging problem, especially in open-world scenarios where categories and contexts are diverse and unpredictable. Traditional methods, however, are typically restricted to closed sets of object categories, limiting their use in complex, open-world scenes. We introduce Open-World Amodal Appearance Completion, a training-free framework that expands amodal completion capabilities by accepting flexible text queries as input. Our approach generalizes to arbitrary objects specified by both direct terms and abstract queries. We term this capability reasoning amodal completion, where the system reconstructs the full appearance of the queried object based on the provided image and language query. Our framework unifies segmentation, occlusion analysis, and inpainting to handle complex occlusions and generates completed objects as RGBA elements, enabling seamless integration into applications such as 3D reconstruction and image editing. Extensive evaluations demonstrate the effectiveness of our approach in generalizing to novel objects and occlusions, establishing a new benchmark for amodal completion in open-world settings. The code and datasets will be released after paper acceptance.

Jiayang Ao, Yanbei Jiang, Qiuhong Ke, Krista A. Ehinger• 2024

Related benchmarks

TaskDatasetResultRank
3D Amodal CompletionExtremeOcc-3D
CLIP Image Similarity0.76
6
Amodal CompletionHiFi-Amodal dataset
CLIP-I87.014
5
Occluded Object RecognitionOccluded COCO
Top-1 Accuracy40.5
5
Occluded Object RecognitionCOCO Separated
Top-1 Acc27.83
5
Amodal CompletionPix2Gestalt ground-truth benchmark (test)
GT-LPIPS0.311
5
Amodal appearance completionVG
CLIP Score28.47
4
Amodal appearance completionCOCO-A
CLIP Score27.612
4
Amodal appearance completionFree Images
CLIP Score28.652
4
Amodal appearance completionLAION
CLIP Score28.123
4
Amodal appearance completionOverall
CLIP Score28.181
4
Showing 10 of 31 rows

Other info

Follow for update