Clevr
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Concept Attribution | CLEVR (test) | F1 Score0.85 | 160 | |
| Visual Question Answering | CLEVR (test) | Overall Accuracy99.8 | 61 | |
| Visual Question Answering | CLEVR 1.0 (test) | Overall Accuracy99.6 | 46 | |
| Object Hallucination Detection | CLEVR | AUROC77.72 | 35 | |
| Criteria-conditioned Clustering | Clevr-4c OpenSMC | Clustering Accuracy (CAcc)84.7 | 30 | |
| Conditional Image Generation | CLEVR | Accuracy98 | 25 | |
| Visual Question Answering | Super-CLEVR (test) | Accuracy87.4 | 24 | |
| Visual Question Answering | CLEVR Style-transferred Target | Accuracy95.9 | 24 | |
| Image Classification | CLEVR | Accuracy76.2 | 22 | |
| Visual Question Answering | CLEVR-Humans 1.0 (test) | Accuracy85.5 | 22 | |
| Visual Question Answering | CLEVR (val) | Overall Accuracy100 | 21 | |
| Concept Attribution | CLEVR | Avg Attribution F151 | 18 | |
| Customized few-shot classification | Clevr 4 10k | Shape Accuracy98.69 | 18 | |
| Unsupervised Object Segmentation | CLEVR 1.0 (test) | FG-ARI95.94 | 16 | |
| Image Classification | CLEVR (test) | Accuracy45.4 | 14 | |
| Visual Reasoning | CLEVR VL-ICL Bench | Average Accuracy38.9 | 14 | |
| Shifted rightmost object color inference | CLEVR | Accuracy (Shifted Rightmost Color)62.06 | 13 | |
| Third object from right color inference | CLEVR | Accuracy41.95 | 13 | |
| Rightmost object material inference | CLEVR | Accuracy86.84 | 13 | |
| Rightmost object shape inference | CLEVR | Accuracy70.03 | 13 | |
| Rightmost object size inference | CLEVR | Accuracy93.22 | 13 | |
| Bottommost object color inference (BC) | CLEVR | BC Accuracy84.09 | 13 | |
| Leftmost object color inference (LC) | CLEVR | Accuracy81.79 | 13 | |
| Rightmost object color inference (RC) | CLEVR | Accuracy98.75 | 13 | |
| Image Generation | CLEVR | FID0.81 | 13 |
Showing 25 of 117 rows