Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts

About

All-in-one image restoration seeks to recover clean images from inputs affected by diverse and unknown degradations using a unified framework. Recent methods have shown strong performance by identifying degradation characteristics to guide the restoration process. However, many of them treat degradations as discrete categories, which limits their ability to model the continuous relational structure that arises in composite degradations. To address this issue, we propose a multimodal large language model (MLLM)-guided image restoration framework that exploits multimodal embeddings as guidance for low-level restoration. Specifically, MLLM-derived features are injected into an encoder-decoder architecture through an MLLM-guided fusion block (MGFB) to enhance degradation-aware representations. In addition, we incorporate a mixture-of-frequency-experts (MoFE) module that adaptively combines frequency experts using MLLM-guided contextual cues. To further improve expert routing, we design an MLLM-guided router with a relational alignment loss that encourages routing patterns consistent with the embedding-space relationships of degraded inputs. Extensive experiments on multiple benchmarks show that the proposed method achieves strong performance across diverse restoration settings and establishes a new state of the art on the challenging CDD11 dataset, outperforming previous methods by up to 1.35 dB.

Eunho Lee, Rei Kawakami, Youngbae Hwang• 2026

Related benchmarks

TaskDatasetResultRank
Image DeblurringGoPro
PSNR32.23
414
DerainingRain100L
PSNR38.54
280
DehazingSOTS
PSNR32.24
238
DenoisingBSD68 sigma=25
PSNR31.59
118
All-in-one Image RestorationSOTS + Rain100L + BSD68 Combined (test)
PSNR32.94
87
Low-light Image EnhancementLOL
PSNR23.4
53
Image RestorationCDD 11 (test)
PSNR (L+H+R)26.13
44
Image RestorationAll-in-one Image Restoration Benchmark (SOTS, Rain100L, BSD68, GoPro, LOLv1) (test)
PSNR (dB)31.55
37
Image DenoisingBSD68
PSNR (sigma=15)34.23
33
Showing 9 of 9 rows

Other info

Follow for update