| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| Spec-Bench | FR-Spec | MT Score195.6 | 48 | 11d ago | |
| gsm8k | HASS | Average Generation Length (τ)5.53 | 31 | 19d ago | |
| MMSPEC 1.0 (test) | MSD | GQA Speedup2.27 | 22 | 1mo ago | |
| SpecBench | ConFu | AVG SR2.73 | 20 | 13d ago | |
| SpecBench v1 (test) | ConFu | WRIT τ4.9 | 12 | 1mo ago | |
| DPR | RASD(PLD) | SR2.49 | 12 | 1mo ago | |
| Qasper | RASD(PLD) | SR1.66 | 12 | 1mo ago | |
| MultiFieldQA | RASD(PLD) | Speculative Rate (SR)2.1 | 12 | 1mo ago | |
| CNN/Daily Mail | RASD(PLD) | SR2.31 | 12 | 1mo ago | |
| HumanEval | RASD(PLD) | Speculative Rate (SR)3 | 12 | 1mo ago | |
| ShareGPT Llama-3.1-8B 1.0 (test) | Eagle_LoRA_ReLU | MT-Bench Score3.2124 | 10 | 1mo ago | |
| LongVideoBench ~15k visual tokens | SpecVLM | Tau (τ)3.82 | 8 | 1mo ago | |
| MVBench | Sparrow | Tau (τ)3.87 | 8 | 1mo ago | |
| VideoDetailCaption ~17k visual tokens | SpecVLM | Tau (τ)3.91 | 8 | 1mo ago | |
| General LLM Evaluation (test) | DEER | Max Accepted Token Length32 | 8 | 1mo ago | |
| Fineweb-edu distillation 8B to 300M | Random Sampling KD | Spec. Accept %62 | 7 | 1mo ago | |
| Fineweb-edu 1.0 (test) | Random Sampling KD (Ours 12+) | Speculative Accept Rate0.735 | 6 | 1mo ago | |
| Spec-Bench OLMo 2 7B | EAGLE-3 + SpecVocab | Conversation Score5.12 | 5 | 1mo ago | |
| OOD | TABEDMTCP | Block Efficiency2.13 | 5 | 1mo ago | |
| Benchmark Second Turn | TABEDMTCP | Block Efficiency2.32 | 5 | 1mo ago | |
| Benchmark First Turn | TABEDMTCP | Block Efficiency2.32 | 5 | 1mo ago | |
| Alpaca | Griffin | Speedup2.98 | 5 | 1mo ago | |
| Qa | Griffin+LTD | Speedup2.23 | 3 | 1mo ago | |
| MT-bench | Griffin+LTD | Speedup2.88 | 3 | 1mo ago | |
| Pre-training dataset | Speculative Accept %0.658 | 3 | 1mo ago |
