Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Orientation Matters: Making 3D Generative Models Orientation-Aligned

About

Humans intuitively perceive object shape and orientation from a single image, guided by strong priors about canonical poses. However, existing 3D generative models often produce misaligned results due to inconsistent training data, limiting their usability in downstream tasks. To address this gap, we introduce the task of orientation-aligned 3D object generation: producing 3D objects from single images with consistent orientations across categories. To facilitate this, we construct Objaverse-OA, a dataset of 14,832 orientation-aligned 3D models spanning 1,008 categories. Leveraging Objaverse-OA, we fine-tune two representative 3D generative models based on multi-view diffusion and 3D variational autoencoder frameworks to produce aligned objects that generalize well to unseen objects across various categories. Experimental results demonstrate the superiority of our method over post-hoc alignment approaches. Furthermore, we showcase downstream applications enabled by our aligned object generation, including zero-shot object orientation estimation via analysis-by-synthesis and efficient arrow-based object rotation manipulation.

Yichong Lu, Yuzhuo Tian, Zijin Jiang, Yikun Zhao, Yuanbo Yang, Hao Ouyang, Haoji Hu, Huimin Yu, Yujun Shen, Yiyi Liao• 2025

Related benchmarks

TaskDatasetResultRank
Novel View SynthesisShapeR (Evaluation Dataset)
PSNR29.4
6
Novel View SynthesisScanNet++ challenging, partially unobserved regions
PSNR26.89
6
Novel View SynthesisShapeR challenging, partially unobserved regions (evaluation)
PSNR27.4
6
Novel View SynthesisScanNet++ (Evaluation)
PSNR24.26
6
3D ReconstructionShapeR Evaluation 40 (real data)
Chamfer Distance (CD)11.29
6
3D ReconstructionScanNet++ real data
Chamfer Distance (CD)11.91
6
3D ReconstructionShapeR Evaluation Dataset (real data)
Chamfer Distance (CD)12.67
6
3D Reconstruction3D-FRONT synthetic data 14
Chamfer Distance (CD)29.07
6
3D ReconstructionScanNet++ real data 49
CD12.5
6
3D Reconstruction3D-Front synthetic data
Chamfer Distance12.57
6
Showing 10 of 12 rows

Other info

Follow for update