Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

OVEN

Benchmarks

Task NameDataset NameSOTA ResultTrend
Knowledge-Based Visual Question AnsweringOVEN (val)
Accuracy (All)25.1
20
Visual Entity RecognitionOVEN
HM (Unseen)28.5
15
Visual Question AnsweringOVEN Query 1.0 (test)
HM30.9
15
Fine-grained Entity RecognitionOVEN Entity 1.0 (test)
HM29.6
15
(Image, Text)-to-Multimodal RetrievalOVEN
R@575.3
14
(Image, Text)-to-Text RetrievalOVEN
Recall@557.8
14
Visual Entity RecognitionOVEN entity (test)
Top-1 Accuracy (Seen)65
11
Open-Vocabulary Entity RecognitionOVEN
EM0.789
8
Multi-modal retrieval (Image-Text to Text/Image-Text)OVEN QS
Recall@58.39
7
Visual Entity RecognitionOVEN (test)
Top-1 Acc (Seen)33.6
7
Multimodal RetrievalOVEN-8
R@575.98
6
Multimodal RetrievalOVEN-6
R@558.17
6
Visual Question AnsweringOVEN
EM15.88
6
Open-domain Visual Entity RecognitionOVEN Wiki (human evaluation set)
Score (Seen Entities)76.1
6
RetrievalOVEN M2KR
R@142.8
4
Image-text-to-multimodal retrievalOVEN M-BEIR (test)
Recall@567.6
4
Image-text-to-text retrievalOVEN M-BEIR (test)
Recall@546.9
4
Open-Vocabulary Entity GroundingOVEN (test)
Accuracy23.1
2
Showing 18 of 18 rows