WonderJourney: Going from Anywhere to Everywhere
About
We introduce WonderJourney, a modularized framework for perpetual 3D scene generation. Unlike prior work on view generation that focuses on a single type of scenes, we start at any user-provided location (by a text description or an image) and generate a journey through a long sequence of diverse yet coherently connected 3D scenes. We leverage an LLM to generate textual descriptions of the scenes in this journey, a text-driven point cloud generation pipeline to make a compelling and coherent sequence of 3D scenes, and a large VLM to verify the generated scenes. We show compelling, diverse visual results across various scene types and styles, forming imaginary "wonderjourneys". Project website: https://kovenyu.com/WonderJourney/
Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T. Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, Charles Herrmann• 2023
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| 3D Scene Generation | WorldScore | Camera Control84.6 | 33 | |
| Video Generation | WorldScore (test) | Average Score54.19 | 12 | |
| 3D Scene Generation | Custom 3D Scene Generation 28 scenes (test) | Time Cost (s)749.5 | 4 | |
| Novel View Rendering | 28 scenes (city, campus, nature, fantasy) (test) | CS Score27.34 | 4 | |
| Perpetual 3D Scene Generation | Custom Nature Evaluation Set | Diversity92.7 | 1 | |
| Perpetual 3D Scene Generation | Custom Multi-style Text-based Evaluation Set | Diversity88.8 | 1 |
Showing 6 of 6 rows