Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

RealOSR: Latent Guidance Boosts Diffusion-based Real-world Omnidirectional Image Super-Resolutions

About

Omnidirectional image super-resolution (ODISR) aims to upscale low-resolution (LR) omnidirectional images (ODIs) to high-resolution (HR), catering to the growing demand for detailed visual content across a $ 180^{\circ}\times360^{\circ}$ viewport. Existing ODISR methods are limited by simplified degradation assumptions (e.g., bicubic downsampling), failing to model and exploit the real-world degradation information. Recent latent-based diffusion approaches using condition guidance suffer from slow inference due to their hundreds of updating steps and frequent use of VAE. To tackle these challenges, we propose \textbf{RealOSR}, a diffusion-based framework tailored for real-world ODISR, featuring efficient latent-based condition guidance within a one-step denoising paradigm. Central to efficient latent-based condition guidance is the proposed \textbf{Latent Gradient Alignment Routing (LaGAR)}, a lightweight module that enables effective pixel-latent space interactions and simulates gradient descent directly in the latent space, thereby leveraging the semantic richness and multi-scale features captured by the denoising UNet. Compared to the recent diffusion-based ODISR method, OmniSSR, RealOSR achieves significant improvements in visual quality and over \textbf{200$\times$} inference acceleration. Our code and models will be released upon acceptance.

Xuhan Sheng, Runyi Li, Bin Chen, Weiqi Li, Xu Jiang, Jian Zhang• 2024

Related benchmarks

TaskDatasetResultRank
Super-ResolutionODI-SR (test)
WS-PSNR22.3
93
Super-ResolutionSUN 360 Panorama (test)
WS-PSNR22.7
70
Super-ResolutionERP images
Inference Time (s)2.36
6
Omnidirectional Image Super-ResolutionSUN 360 severe degradation
WS-PSNR22.25
5
Omni-directional Image Super-ResolutionODI-SR
WS-PSNR22.3
4
Omni-directional Image Super-ResolutionSUN 360
WS-PSNR22.7
4
Showing 6 of 6 rows

Other info

Follow for update