Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DemoFusion: Democratising High-Resolution Image Generation With No $$$

About

High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as "previews", facilitating rapid prompt iteration.

Ruoyi Du, Dongliang Chang, Timothy Hospedales, Yi-Zhe Song, Zhanyu Ma• 2023

Related benchmarks

TaskDatasetResultRank
Text-to-Image GenerationLAION-5B 1,000 prompts
FID (Real)47.079
20
Text-to-Image Generation4K Resolution 4K x 4K (test)
CLIP IQA Score0.4392
16
High-Resolution Image GenerationLAION-5B 3x3 scaling factor (test)
FID68.82
7
High-Resolution Image GenerationLAION-5B 4x4 scaling factor (test)
FID65.89
7
High-Resolution Image GenerationLAION 5B 2x2 scaling factor (test)
FID63.24
7
Image-to-VideoVBench I2V
Average VBench Score0.879
6
Image-to-VideoFrescoArchive
Average VBench Score0.903
6
High-Resolution Image GenerationHigh-resolution Image Generation
FID_r81.69
6
High-Resolution Image GenerationResolution 4096 x 4096
FID74.75
5
Showing 9 of 9 rows

Other info

Code

Follow for update