Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

An Intelligent Agentic System for Complex Image Restoration Problems

About

Real-world image restoration (IR) is inherently complex and often requires combining multiple specialized models to address diverse degradations. Inspired by human problem-solving, we propose AgenticIR, an agentic system that mimics the human approach to image processing by following five key stages: Perception, Scheduling, Execution, Reflection, and Rescheduling. AgenticIR leverages large language models (LLMs) and vision-language models (VLMs) that interact via text generation to dynamically operate a toolbox of IR models. We fine-tune VLMs for image quality analysis and employ LLMs for reasoning, guiding the system step by step. To compensate for LLMs' lack of specific IR knowledge and experience, we introduce a self-exploration method, allowing the LLM to observe and summarize restoration results into referenceable documents. Experiments demonstrate AgenticIR's potential in handling complex IR tasks, representing a promising path toward achieving general intelligence in visual processing.

Kaiwen Zhu, Jinjin Gu, Zhiyuan You, Yu Qiao, Chao Dong• 2024

Related benchmarks

TaskDatasetResultRank
Image DenoisingBSD68
PSNR12.35
404
DerainingRain100L
PSNR35.04
196
Image DehazingSOTS Outdoor
PSNR20.43
124
Image DehazingSOTS Indoor
PSNR33.46
83
Low-light Image EnhancementLOL v1.0 (test)
PSNR9.32
35
DesnowingWeatherBench
PSNR20.35
33
Image RestorationCDD 11 (test)
PSNR (Low)10.62
29
Image RestorationMiO100 AgenticIR setting (Group A)
PSNR21.04
24
Image RestorationMiO100 AgenticIR setting (Group C)
PSNR18.82
24
Image RestorationMiO100 AgenticIR setting (Group B)
PSNR20.55
24
Showing 10 of 45 rows

Other info

Follow for update