Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FineGen: A VLM-based Multi-Agent Framework for Fine-Grained Image-Text Dataset Construction

About

The scarcity of hard negative samples in current vision-language datasets significantly hinders fine-grained perception. To address this, we propose FineGen, a VLM-based Multi-Agent framework for automated dataset construction. By employing a collaborative Generation-Verification-Correction pipeline with a closed-loop feedback mechanism, FineGen ensures synthesized hard negatives are semantically valid yet strictly contradictory to visual content. Applying this to ImageNet, we construct FineGen-100K, a hierarchical dataset containing over 147,000 attribute-specific hard negatives with a rigorous 1:10 positive-to-negative ratio. Extensive evaluations confirm a 96.7% attribute validity rate. Crucially, downstream validation on the FG-OVD benchmark shows that fine-tuning on FineGen-100K yields a substantial +14.4% accuracy improvement on hard samples, significantly outperforming state-of-the-art methods.

Chang Kong, Yuebing Li, Peng Mo, Haigang Zhang, Qiuming Luo• 2026

Related benchmarks

TaskDatasetResultRank
Fine-grained image-level matchingFG-OVD (Hard)--
6
Fine-grained image-level matchingFG-OVD (Medium)--
6
Fine-grained image-level matchingFG-OVD Easy--
6
Fine-grained image-level matchingFG-OVD (Trivial)--
6
Showing 4 of 4 rows

Other info

Follow for update