NLVR2
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Natural Language Visual Reasoning | NLVR2 (test-p) | Accuracy92.6 | 346 | |
| Natural Language Visual Reasoning | NLVR2 (dev) | Accuracy91.51 | 307 | |
| Visual Reasoning | NLVR2 | Accuracy92.6 | 49 | |
| Visual Reasoning | NLVR2 (test) | Accuracy85.15 | 46 | |
| Adversarial Attack | NLVR2 | Attack Success Rate67.51 | 37 | |
| Natural Language Visual Reasoning | NLVR2 | Accuracy87.3 | 21 | |
| Visual Reasoning | NLVR2 (test-P) | Accuracy92.6 | 21 | |
| Visual reasoning | NLVR2 (val) | Accuracy91.1 | 20 | |
| Visual Reasoning | NLVR2 v2 (dev) | Accuracy88.7 | 20 | |
| Visual Reasoning | NLVR2 (dev) | Accuracy82.5 | 18 | |
| Natural Language Visual Reasoning | NLVR2 (test) | Accuracy85.36 | 16 | |
| Natural Language Visual Reasoning | NLVR2 std | Accuracy85.5 | 14 | |
| Visual Reasoning | NLVR2 (test-dev) | Accuracy79.87 | 14 | |
| Multi-image reasoning and general capability evaluation | NLVR2 | Accuracy90.42 | 12 | |
| Natural Language Visual Reasoning | NLVR2 (val) | Accuracy83.15 | 12 | |
| Multi-image Understanding | NLVR2 (test) | Accuracy87.3 | 9 | |
| Visual Reasoning | NLVR2 loc (val) | Accuracy77.27 | 5 | |
| Visual Reasoning | NLVR2 | GFLOPs20.01 | 4 | |
| Natural Language Visual Reasoning | NLVR2 | GFLOPs17.4 | 4 | |
| Natural Language Visual Reasoning | NLVR2 (test-u) | Accuracy67.3 | 2 |
Showing 20 of 20 rows