Macro Average
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| General Task Performance | Macro-average (Mathematics, Multi-Hop QA, Code Generation) | Accuracy69 | 21 | |
| Re-identification | Macro Average (Across Datasets) | AUC96.9 | 18 | |
| Mathematical Reasoning | Macro Average AIME2024, MATH, Minerva, Olympiad-Bench | Pass@155 | 16 | |
| Mathematical Reasoning | Macro Average Selected Benchmarks | Pass@1 (Avg@32)52.8 | 14 | |
| Regression | Macro-average SICKR-STS, STS-B, WMT_RU_EN, WMT_EN_ZH, WMT_SI_EN (test) | Pearson Correlation (r)76.3 | 11 | |
| Question Answering and Reasoning | Macro-average (MMLU, MATH, GSM8K, BBH) | Cost Reduction46 | 8 |
Showing 6 of 6 rows