Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Multi-View Pose-Agnostic Change Localization with Zero Labels

About

Autonomous agents often require accurate methods for detecting and localizing changes in their environment, particularly when observations are captured from unconstrained and inconsistent viewpoints. We propose a novel label-free, pose-agnostic change detection method that integrates information from multiple viewpoints to construct a change-aware 3D Gaussian Splatting (3DGS) representation of the scene. With as few as 5 images of the post-change scene, our approach can learn an additional change channel in a 3DGS and produce change masks that outperform single-view techniques. Our change-aware 3D scene representation additionally enables the generation of accurate change masks for unseen viewpoints. Experimental results demonstrate state-of-the-art performance in complex multi-object scenes, achieving a 1.7x and 1.5x improvement in Mean Intersection Over Union and F1 score respectively over other baselines. We also contribute a new real-world dataset to benchmark change detection in diverse challenging scenes in the presence of lighting variations.

Chamuditha Jayanga Galappaththige, Jason Lai, Lloyd Windrim, Donald Dansereau, Niko Suenderhauf, Dimity Miller• 2024

Related benchmarks

TaskDatasetResultRank
Scene Change DetectionPASLCD (test)
mIoU47.8
14
Change DetectionPASLCD Cantina 1.0 (Indoor)
mIoU58
6
Change DetectionPASLCD Lounge Indoor 1.0
mIoU46.3
6
Change DetectionPASLCD Printing Area Indoor 1.0
mIoU58.8
6
Change DetectionPASLCD Lunch Room Indoor 1.0
mIoU0.389
6
Change DetectionPASLCD Meeting Room Indoor 1.0
mIoU35
6
Change DetectionPASLCD Garden Outdoor 1.0
mIoU43.6
6
Change DetectionPASLCD Pots Outdoor 1.0
mIoU0.54
6
Change DetectionPASLCD Zen Outdoor 1.0
mIoU50
6
Change DetectionPASLCD Playground 1.0 (Outdoor)
mIoU24.9
6
Showing 10 of 19 rows

Other info

Code

Follow for update