UltraDomain
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Retrieval-Augmented Generation | UltraDomain mix | Win Rate76.2 | 18 | |
| Retrieval-Augmented Generation | UltraDomain agriculture | Win Rate95 | 18 | |
| Question Answering | UltraDomain Mix 1.0 (512 question-answer pairs) | Exact Match (EM)54.49 | 10 | |
| Question Answering | UltraDomain Legal 1.0 | EM35.16 | 10 | |
| Question Answering | UltraDomain Computer Science 1.0 | EM41.02 | 10 | |
| Question Answering | UltraDomain Agriculture 1.0 | Exact Match (EM)34.96 | 10 | |
| Long-text Question Answering | UltraDomain | F1 (bio)36.19 | 10 | |
| Conciseness Evaluation | UltraDomain | Bio Score100 | 9 | |
| Faithfulness Evaluation | UltraDomain | Bio Score100 | 9 | |
| Retrieval-Augmented Generation | UltraDomain Pathology (test) | Comprehension92.35 | 9 | |
| Retrieval-Augmented Generation | UltraDomain Mix (test) | Comprehensiveness91.24 | 9 | |
| Global Question Answering | UltraDomain Mix | Comprehensiveness76.8 | 6 | |
| Global Question Answering | UltraDomain Legal | Comprehensiveness60.8 | 6 | |
| Global Question Answering | UltraDomain Computer Science | Comprehensiveness68.8 | 6 | |
| Global Question Answering | UltraDomain Agriculture | Comprehensiveness65.6 | 6 | |
| Knowledge Graph Question Answering | UltraDomain Legal | Comprehensive Score86.19 | 5 | |
| Knowledge Graph Question Answering | UltraDomain History | Comprehensive87.75 | 5 | |
| Knowledge Graph Question Answering | UltraDomain Cooking | Comprehensive Score89.25 | 5 | |
| Knowledge Graph Question Answering | UltraDomain Biology | Comprehensive89.44 | 5 | |
| Knowledge Graph Question Answering | UltraDomain Agriculture | Comprehensive Score89.93 | 5 | |
| General QA | UltraDomain Physics (test) | Comprehensiveness89.51 | 5 | |
| General QA | UltraDomain Philosophy (test) | Comprehensive86.78 | 5 | |
| General QA | UltraDomain Music (test) | Comprehensiveness87.95 | 5 | |
| General QA | UltraDomain Mix (test) | Comprehensive Score87.15 | 5 | |
| General QA | UltraDomain Mathematics (test) | Comprehensiveness87.3 | 5 |
Showing 25 of 31 rows