DataComp
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Zero-shot Image Classification | DataComp | DataComp Score63.47 | 28 | |
| Classification and Retrieval | DataComp small 38 downstream tasks | Mean Score17.3 | 12 | |
| Image Classification | DataComp 38 downstream tasks medium (test) | Accuracy66.7 | 12 | |
| Zero-shot classification | DataComp (test) | DataComp Average57.34 | 10 | |
| Multi-modal Representation Learning | DataComp medium Evaluation Suite | Average Score37.7 | 9 | |
| Zero-shot Evaluation | DataComp full (test) | Mean Zero-shot Accuracy63.47 | 8 | |
| Data Membership Prediction | DataComp | Jaccard Similarity27.1 | 6 | |
| Zero-shot Image Classification | DataComp Medium scale 128M sample pool | Average Accuracy (38 Datasets)32.3 | 5 | |
| Zero-shot Image Classification and Retrieval | DataComp small 10M (train) | Average Zero-Shot Performance14.6 | 5 | |
| Multi-modal Evaluation | DataComp | Datacomp Average31.89 | 5 | |
| Multi-modal Classification | DataComp Evaluation Suite 1.0 | DataComp Average39.16 | 5 | |
| Image Classification | DataComp | Average Score25.08 | 5 | |
| Classification | Datacomp DFN-1B track | Average Performance (Datacomp)57.34 | 5 | |
| Classification | Datacomp DFN-192M track | Average Performance (Datacomp)54.9 | 5 | |
| Classification | Datacomp DFN-14M track | Average Performance39.16 | 5 | |
| Classification | Datacomp CC12M track | Average Performance31.89 | 5 | |
| Classification | Datacomp CC3M track | Average Performance25.08 | 5 | |
| Zero-shot Image Classification | DataComp Small scale 12.8M sample pool | Average Accuracy (Zero-shot)16.5 | 4 | |
| Text-to-Image Retrieval | DataComp 15M | Accuracy74 | 4 | |
| Zero-shot classification | DataComp medium-scale (test) | Average Score26.8 | 3 | |
| Zero-shot Image Classification and Retrieval | DataComp medium 100M (train) | Zero-shot Performance25.8 | 2 |
Showing 21 of 21 rows