Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Winoground

Benchmarks

Task NameDataset NameSOTA ResultTrend
Compositional Vision-Language ReasoningWinoground
Text Score89.5
47
Compositional Scene UnderstandingWinoground
Text Alignment Score64
29
Image-Text MatchingWinoground
Text Agreement Score89.5
26
Compositional ReasoningWinoground
Txt2Img Score40.25
21
Visual Question AnsweringWinogroundVQA v1.0 (test)
Accuracy46.5
14
Fine-grained retrievalWinoground (test)
Text Agreement (%)40
12
Image-text alignmentWinoground (test)
Text Score89.5
12
Fine-grained Image-Text MatchingWinoground
Group Agreement25.8
11
Vision-Language ReasoningWinoground
Simple Acc59.88
9
Text-to-image retrievalWinoground
R@1 (T2I)0.133
8
Vision-Language Compositional ReasoningWinoground standard (test)
Text Score75.5
7
Text SelectionWinoground
Text Score34
7
Image SelectionWinoground
Image Score14
7
Image-Text MatchingWinoground 1.0 (full)
Text Agreement Score89.5
5
Vision-Language ReasoningWinoground
Text Score30.5
4
Compositional EvaluationWinoground Txt2Img
Txt2Img Score14
4
Vision-Language Compositional ReasoningWinoground (test)
Object Score0.461
4
Image-Text MatchingWinoground clean
Text Agreement Score52.63
4
Image-Text MatchingWinoground (full)
Accuracy52.7
3
Vision-Language Compositional ReasoningWinoground 1.0 (test)
Text Score42.5
3
Compositional ReasoningWinoground (test)
Image Accuracy27
3
Paired-prompt evaluationWinoground
Simple Accuracy58.81
2
Compositional ReasoningWinoground clean 171 samples
Text Score31.58
2
Vision-Language Compositional ReasoningWinoground clean (no-tag)
Text Score32.16
2
Showing 24 of 24 rows