SciKnowEval
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Scientific Reasoning | SciKnowEval | Chemistry Accuracy82.38 | 56 | |
| Scientific Reasoning | SciKnowEval Chemistry | mean@1694.3 | 27 | |
| Scientific Reasoning | SciKnowEval Physics | Mean@1663.39 | 24 | |
| Scientific Reasoning | SciKnowEval Material | Mean@1671.48 | 24 | |
| Scientific Reasoning | SciKnowEval Biology | Mean@1637.08 | 24 | |
| Scientific Reasoning | SciKnowEval L3 | Chemistry Score81 | 18 | |
| Scientific Question Answering | SciKnowEval | Accuracy (Biology)68.5 | 15 | |
| Scientific Knowledge Evaluation | SciKnowEval | Accuracy52.1 | 12 | |
| Calibration | SciKnowEval Edu. MMLU-Pro (test) | Average Calibration Score21.1 | 9 | |
| Calibration | SciKnowEval (test) | BIO Score0.226 | 9 | |
| Multi-domain Science Q&A | SciKnowEval Phys-Chem (test) | Accuracy (Physics)58.8 | 8 | |
| Multi-domain Science Q&A | SciKnowEval Chem-Mat (test) | Chemistry Accuracy71.3 | 8 | |
| Multi-domain Science Q&A | SciKnowEval Mat-Phys (test) | Accuracy (Mat. Sci.)70.5 | 8 | |
| Scientific Reasoning | SciKnowEval reasoning 8 | Chemistry Score79.61 | 6 | |
| Scientific reasoning | SciKnowEval Hard | Pass@142.4 | 5 | |
| Scientific reasoning | SciKnowEval Hardest | Pass@121 | 5 | |
| Scientific Reasoning | SciKnowEval (test) | Biology Accuracy65.38 | 4 | |
| Scientific Reasoning | SciKnowEval Physics (val) | Best@1693.8 | 3 | |
| Scientific Reasoning | SciKnowEval Material (val) | Best@1687.9 | 3 | |
| Scientific Reasoning | SciKnowEval Biology (val) | Best@1680.1 | 3 | |
| Multiple-choice reasoning | SciKnowEval Chemistry L3 (test) | Final Test Mean Score@1680.6 | 3 |
Showing 21 of 21 rows