RareBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Rare Disease Diagnosis | RareBench RAMEDIS | Recall@173.23 | 21 | |
| Rare Disease Diagnosis | RareBench LIRICAL | R@155.95 | 21 | |
| Rare Disease Diagnosis | RareBench HMS | Recall@156.97 | 21 | |
| Rare Disease Diagnosis | RareBench MME | R@177.5 | 21 | |
| Medical Diagnosis | RareBench Dirichlet α=0.3 partition | Accuracy68 | 18 | |
| Text-to-Image Generation | RareBench (test) | Property Score97.5 | 15 | |
| Text-to-image alignment | RareBench v1 (test) | Property Alignment96.9 | 11 | |
| Text-to-image alignment | RareBench | Property (Single Object)96.3 | 11 | |
| Rare disease diagnosis from phenotypes | RareBench-Public Phenotypes | Accuracy37.92 | 9 | |
| Clinical diagnosis | RareBench HMS subset n=88 | Recall@151.14 | 7 | |
| Clinical diagnosis retrieval | RareBench MME n=40 | R@140 | 7 | |
| Clinical diagnosis | RareBench (combined) | Recall@139.31 | 7 | |
| Clinical Diagnosis | RareBench LIRICAL | Recall@136.24 | 7 | |
| Rare Disease Diagnosis | RareBench Qwen-0.8B text (test) | Hit@185.6 | 5 | |
| Text-to-Image Generation | RareBench | Latency (sec)7.36 | 4 | |
| Long-tail Recognition | RareBench | Conditional Hit@185.6 | 4 | |
| Rare Disease Diagnosis | RareBench | Pass@146.79 | 4 | |
| Text-to-Image Generation | RareBench Overall | Pick Score74.5 | 3 | |
| Text-to-Image Generation | RareBench Advanced | Pick71.5 | 3 | |
| Text-to-Image Generation | RareBench Basic | Pick Score95.5 | 3 | |
| Text-to-Image Generation | RareBench | Property Score76.7 | 3 | |
| Text-to-Image Synthesis | RareBench (full) | LAION-aesthetic6.395 | 3 | |
| Text-to-Image Generation | RareBench 1.0 (test) | Property Score81.9 | 3 | |
| Biomedical Question Answering | RareBench | Metric- | 0 |
Showing 24 of 24 rows