M-BEIR
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multi-modal Retrieval | M-BEIR (test) | Average Recall69.7 | 61 | |
| Multi-modal Retrieval | M-BEIR Global Pool 1.0 (test) | VisualNews R@5 (qt->ci)48.8 | 11 | |
| Text-to-Image Retrieval | M-BEIR Flickr30k | R@179.1 | 8 | |
| Text-to-Image Retrieval | M-BEIR COCO | Recall@155.2 | 8 | |
| Multi-modal Information Retrieval | M-BEIR local pool setting | q_t_to_c_i VN R@549.4 | 8 | |
| Multimodal Retrieval | M-BEIR extended (test) | Recall (I→I)32.1 | 7 | |
| Multimodal-to-Text Retrieval | M-BEIR InfoSeek | R@114.5 | 4 | |
| Multimodal-to-Multimodal Retrieval | M-BEIR OVEN | Recall@149.4 | 4 | |
| Multimodal-to-Image Retrieval | M-BEIR CIRR | Recall@124.4 | 4 | |
| Image-to-Image Retrieval | M-BEIR NIGHTS | Recall@18.8 | 4 | |
| Image-to-Text Retrieval | M-BEIR COCO i→t | R@171.1 | 4 | |
| Text-to-Multimodal Retrieval | M-BEIR WebQA (t→i,t) | R@149.5 | 4 | |
| Text-to-Text Retrieval | M-BEIR WebQA (t→t) | Recall@158.3 | 4 |
Showing 13 of 13 rows