Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Compositional Reasoning on SugarCrepe 1.0 (test)
Loading...
100
Replace Acc (Object)
Human
46.96
60.73
74.5
88.27
Dec 19, 2024
Replace Acc (Object)
Replace Acc (Attribute)
Replace Acc (Relation)
Swap Acc (Object)
Swap Acc (Attribute)
Add Acc (Object)
Add Acc (Attribute)
Updated 4d ago
Evaluation Results
Method
Method
Links
Replace Acc (Object)
Replace Acc (Attribute)
Replace Acc (Relation)
Swap Acc (Object)
Swap Acc (Attribute)
Add Acc (Object)
Add Acc (Attribute)
Human
2024.12
100
99
97
99
100
99
99
LAION
backbone=xlm-roberta-l...
2024.12
97
86
72
64
72
93
86
Ours (Grounded Recaptioning)
training_data=64M synt...
2024.12
97
94
88
89
94
95
93
DC-XL
2024.12
96
85
70
65
67
91
85
GPT-4V
sees both captions sim...
2024.12
96
94
90
83
90
92
92
CLIP
2024.12
94
79
65
60
62
78
72
CapPa
objective=captioning
2024.12
92
90
87
82
88
99
99
Vera
text-only=true, sees i...
2024.12
49
50
49
49
49
49
50
Feedback
Search any
task
Search any
task
