NeAR: Coupled Neural Asset-Renderer Stack
About
Neural asset authoring and neural rendering have traditionally evolved as disjoint paradigms: one generates digital assets for fixed graphics pipelines, while the other maps conventional assets to images. However, treating them as independent entities limits the potential for end-to-end optimization in fidelity and consistency. In this paper, we bridge this gap with NeAR, a Coupled Neural Asset--Renderer Stack. We argue that co-designing the asset representation and the renderer creates a robust "contract" for superior generation. On the asset side, we introduce the Lighting-Homogenized SLAT (LH-SLAT). Leveraging a rectified-flow model, NeAR lifts casually lit single images into a canonical, illumination-invariant latent space, effectively suppressing baked-in shadows and highlights. On the renderer side, we design a lighting-aware neural decoder tailored to interpret these homogenized latents. Conditioned on HDR environment maps and camera views, it synthesizes relightable 3D Gaussian splats in real-time without per-object optimization. We validate NeAR on four tasks: (1) G-buffer-based forward rendering, (2) random-lit reconstruction, (3) unknown-lit relighting, and (4) novel-view relighting. Extensive experiments demonstrate that our coupled stack outperforms state-of-the-art baselines in both quantitative metrics and perceptual quality. We hope this coupled asset-renderer perspective inspires future graphics stacks that view neural assets and renderers as co-designed components instead of independent entities.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Reflective Object Reconstruction | Glossy Synthetic | PSNR25.32 | 19 | |
| Relighting | ADT | LPIPS0.0693 | 9 | |
| Relighting | DTC | LPIPS0.0475 | 9 | |
| Relighting | Glossy Synthetic | LPIPS0.0465 | 9 | |
| Relighting | Objaverse | LPIPS0.0486 | 9 | |
| Dynamic Reconstruction | ADT | PSNR22.89 | 7 | |
| Reconstruction | DTC | LPIPS0.0532 | 4 | |
| Reconstruction | Objaverse | LPIPS0.0394 | 4 | |
| Forward Rendering | ADT | LPIPS0.0488 | 2 | |
| Forward Rendering | DTC | LPIPS0.0458 | 2 |