CMNIST
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Graph Classification | CMNIST-75sp unbiased (test) | Accuracy82.73 | 60 | |
| Image Classification | CMNIST (test) | Test Accuracy93.54 | 55 | |
| Robust Image Classification | cMNIST (test) | WGA85.26 | 33 | |
| Image Classification | cMNIST | WGA75.75 | 26 | |
| Image Classification | CMNIST 0.5% bias ratio unbiased (test) | Accuracy97.52 | 17 | |
| Graph Classification | CMNIST color | Accuracy36.37 | 17 | |
| Graph Classification | CMNIST-sp Superpixel distribution shift | Accuracy68.94 | 15 | |
| Graph Classification | CMNIST color OOD | Accuracy48.1 | 14 | |
| Graph Classification | CMNIST color ID | Accuracy79.3 | 14 | |
| Graph Classification | CMNIST color GOOD | Accuracy53.3 | 14 | |
| Group Robustness | cMNIST (test) | Worst Group Accuracy (WGA)83.37 | 13 | |
| Graph Classification | CMNIST GraphOOD (test) | Accuracy52.17 | 13 | |
| Graph Classification | CMNIST | Accuracy28.49 | 13 | |
| Subpopulation Shift | CMNIST | AA69.66 | 12 | |
| Graph Classification | CMNIST-SP | Accuracy62.13 | 12 | |
| Image Classification | CMNIST 5% bias ratio unbiased (test) | Accuracy98.13 | 11 | |
| Image Classification | CMNIST 2% bias ratio unbiased (test) | Accuracy97.15 | 11 | |
| Image Classification | CMNIST 1% bias ratio unbiased (test) | Accuracy95.35 | 11 | |
| Graph Classification | CMNIST correlation shift (test) | Accuracy49.73 | 10 | |
| Image Classification | CMNIST original unshifted | Worst Accuracy73.6 | 9 | |
| Domain Adaptation | CMNIST v1 (test) | Accuracy (Shift 1.0)100 | 9 | |
| Binary Classification | CMNIST (test) | WGA94.3 | 8 | |
| Graph Classification | CMNIST (test) | Accuracy69 | 8 | |
| Image Classification | CMNIST In-distribution | Accuracy99.5 | 7 | |
| Unsupervised Disentanglement | CMNIST (test) | DCI0.948 | 7 |
Showing 25 of 32 rows