Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

OVEN

Benchmarks

Task NameDataset NameSOTA ResultTrend
Visual Question AnsweringOVEN Query 1.0 (test)
HM30.9
15
Fine-grained Entity RecognitionOVEN Entity 1.0 (test)
HM29.6
15
(Image, Text)-to-Multimodal RetrievalOVEN
R@575.3
14
(Image, Text)-to-Text RetrievalOVEN
Recall@557.8
14
Open-Vocabulary Entity RecognitionOVEN
EM0.789
8
Multi-modal retrieval (Image-Text to Text/Image-Text)OVEN QS
Recall@58.39
7
Visual Entity RecognitionOVEN (test)
Top-1 Acc (Seen)33.6
7
Multimodal RetrievalOVEN-8
R@575.98
6
Multimodal RetrievalOVEN-6
R@558.17
6
Visual Question AnsweringOVEN
EM15.88
6
Open-domain Visual Entity RecognitionOVEN Wiki (human evaluation set)
Score (Seen Entities)76.1
6
Image-text-to-multimodal retrievalOVEN M-BEIR (test)
Recall@567.6
4
Image-text-to-text retrievalOVEN M-BEIR (test)
Recall@546.9
4
Open-Vocabulary Entity GroundingOVEN (test)
Accuracy23.1
2
Showing 14 of 14 rows