Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Waterbirds

Benchmarks

Task NameDataset NameSOTA ResultTrend
Image ClassificationWaterbirds
Average Accuracy99.2
157
Image ClassificationWaterbirds In-context training: iNaturalist (test)
Average Worst-Group Accuracy92.23
140
Image ClassificationWaterbirds (test)
Worst-Group Accuracy93
112
Image ClassificationWaterbirds severe in-distribution context (test)
Worst-group Accuracy91.38
104
Image ClassificationWaterbirds In-context training (test)
Average Worst-Group Accuracy93.4
104
ClassificationWaterbirds Background (test)
Accuracy91.8
24
Zero-shot classification fairnessWaterbirds Background
Accuracy (Zero-shot)88.6
24
Image ClassificationWaterbirds 100%
Accuracy Variance across Groups16
22
Image ClassificationWaterbirds 95%
Accuracy Variance (Group)6
22
Object ClassificationWaterbirds (test)
Worst-Group Accuracy90
22
Image ClassificationWaterbirds
Relative Accuracy Improvement16.68
18
ClassificationWaterbirds (test)
Test Accuracy90.6
15
Image ClassificationWaterbirds Flip (test)
Accuracy90.6
14
Image ClassificationWaterbirds Original (test)
Accuracy98.4
14
Group RobustnessWaterbirds 100%
Worst Group Accuracy79.7
11
Group RobustnessWaterbirds 95%
Worst Group Accuracy89.7
11
Image ClassificationWaterbirds 100% correlation (test)
Worst-group Accuracy79.7
11
Image ClassificationWaterbirds 95% correlation (test)
Worst-group Accuracy89.7
11
Slice discovery and debiasingWaterbirds
Worst-group Accuracy92.4
10
Image ClassificationWaterbirds original unshifted
Worst Accuracy90.8
10
Binary ClassificationWaterbirds CB
Accuracy77.9
10
Image ClassificationWaterbirds (test)
Avg Acc (0.5% Bias)63.64
10
ClassificationWaterbirds 5.0 severity (test)
Accuracy66.33
10
ClassificationWaterbirds 2.0 severity (test)
Accuracy65.23
10
ClassificationWaterbirds severity 1.0 (test)
Accuracy0.6522
10
Showing 25 of 44 rows