Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Colored MNIST

Benchmarks

Task NameDataset NameSOTA ResultTrend
Image ClassificationColored-MNIST (CMNIST)
Accuracy94.43
42
Image classificationColored MNIST unbiased (test)
Accuracy98.02
28
Digit ClassificationColored MNIST foreground color (test)
Unbiased Accuracy86.37
24
Image ClassificationColored MNIST (test)
Accuracy75
24
Image ClassificationColored MNIST (unbiased)
Accuracy0.9628
16
Image ClassificationColored MNIST (bias-guiding)
Accuracy100
16
Image ClassificationColored MNIST (train)
Accuracy88.9
14
Image ClassificationColored MNIST
Accuracy (+90% Threshold)73.1
9
Image ClassificationColored MNIST background color, ratio 0.95
Bias-Aligned Accuracy99.9
5
Image ClassificationColored MNIST background color, ratio 0.98
Bias-Aligned Accuracy99.97
5
Image ClassificationColored MNIST background color, ratio 0.99
Bias Aligned Score100
5
Image ClassificationColored MNIST background color, ratio 0.995
Aligned Accuracy1
5
Digit ClassificationColored MNIST (val)
Validation Accuracy (Kendall's tau)1
5
Color ClassificationColored-MNIST (val)
Unbiased Accuracy99.95
4
OOD DetectionColored MNIST In-distribution vs Textures protocol (test)
TNR @ 95% TPR100
2
OOD DetectionColored MNIST In-distribution vs iSUN protocol (test)
TNR @ 95% TPR100
2
OOD DetectionColored MNIST In-distribution vs LSUN protocol (test)
TNR @ 95% TPR1
2
OOD DetectionColored MNIST In-distribution vs Spurious OOD protocol (test)
TNR @ 95% TPR0.958
2
Showing 18 of 18 rows