Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

An Intelligent Agentic System for Complex Image Restoration Problems

About

Real-world image restoration (IR) is inherently complex and often requires combining multiple specialized models to address diverse degradations. Inspired by human problem-solving, we propose AgenticIR, an agentic system that mimics the human approach to image processing by following five key stages: Perception, Scheduling, Execution, Reflection, and Rescheduling. AgenticIR leverages large language models (LLMs) and vision-language models (VLMs) that interact via text generation to dynamically operate a toolbox of IR models. We fine-tune VLMs for image quality analysis and employ LLMs for reasoning, guiding the system step by step. To compensate for LLMs' lack of specific IR knowledge and experience, we introduce a self-exploration method, allowing the LLM to observe and summarize restoration results into referenceable documents. Experiments demonstrate AgenticIR's potential in handling complex IR tasks, representing a promising path toward achieving general intelligence in visual processing.

Kaiwen Zhu, Jinjin Gu, Zhiyuan You, Yu Qiao, Chao Dong• 2024

Related benchmarks

TaskDatasetResultRank
Image DenoisingBSD68
PSNR12.35
419
DerainingRain100L
PSNR35.04
280
Image DehazingSOTS Outdoor
PSNR20.43
124
Image DehazingSOTS Indoor
PSNR33.46
83
Image RestorationMiO100 AgenticIR setting (Group A)
PSNR21.04
47
Image RestorationMiO100 AgenticIR setting (Group B)
PSNR20.55
47
Image RestorationCDD 11 (test)
PSNR (L+H+R)11.07
44
Low-light Image EnhancementLOL v1.0 (test)
PSNR9.32
44
Image RestorationMiO100 AgenticIR setting (Group C)
PSNR18.82
35
DesnowingWeatherBench
PSNR20.35
33
Showing 10 of 53 rows

Other info

Follow for update