| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| gsm8k | TreeFlash | Average Generation Length (τ)9.43 | 109 | 1mo ago | |
| MT-bench | DDTree | Tau (τ)6.06 | 81 | 1mo ago | |
| LiveCodeBench | TAPS | Speedup Factor7.16 | 66 | 1mo ago | |
| Spec-Bench | FR-Spec | MT Score195.6 | 57 | 1mo ago | |
| MBPP | TAPS | Speedup6.75 | 52 | 1mo ago | |
| MATH 500 | TreeFlash | Speedup7.99 | 52 | 1mo ago | |
| HumanEval | TAPS | Speedup Factor6.99 | 52 | 1mo ago | |
| GPQA Diamond | STAND | Throughput91.17 | 48 | 2mo ago | |
| AIME 2024 | STAND | Throughput (T)69.15 | 48 | 2mo ago | |
| SpecBench | MicroSpec | AVG SR900.7 | 47 | 1mo ago | |
| Avg. | TAPS | Speedup6.73 | 32 | 29d ago | |
| SQL | HedgeSpec | MAT8.06 | 30 | 3mo ago | |
| CNN_DM | LLaMA-3.1-8B-IT | MAT1 | 30 | 3mo ago | |
| MedQA | HedgeSpec | Match Rate (MAT)6.47 | 30 | 3mo ago | |
| Chemistry | HedgeSpec | MAT7.1 | 30 | 3mo ago | |
| Biology | HedgeSpec | MAT7.18 | 30 | 3mo ago | |
| Math | HedgeSpec | Match Rate7.69 | 30 | 3mo ago | |
| Python | HedgeSpec | MAT7.69 | 30 | 3mo ago | |
| MT-Bench, HumanEval, and GSM8K Mean | MTP Lλ LK | Mean Acceptance Length (tau)4.83 | 26 | 1mo ago | |
| AIME 25 | TAPS | Speedup7.08 | 26 | 1mo ago | |
| Med | EvoSpec | Throughput (tokens/s)128.51 | 22 | 1mo ago | |
| Law | EvoSpec | Throughput (tokens/s)132.69 | 22 | 1mo ago | |
| Code | EvoSpec | Throughput (tokens/s)138.72 | 22 | 1mo ago | |
| MMSPEC 1.0 (test) | MSD | GQA Speedup2.27 | 22 | 4mo ago | |
| 20 Prompts across 4 Task Categories | SpecKV-acc. | Mean Expected Tokens per Speculation Step6.55 | 20 | 2mo ago |
