Share your thoughts, 1 month free Claude Pro on usSee more

Speculative Decoding on MMSPEC 1.0 (test)

2.27GQA Speedup

MSD

Updated 1mo ago

Evaluation Results

Method	Links
MSD 2026.03		2.27	2.32	2.37	1.8	2.35	2.03	2.67	1.96	2.42	4.06	2.51	2.78	2.57	2.58
MSD 2026.03		1.92	3.62	3.68	1.88	3.7	2.48	3.74	1.97	3.49	2.47	3.68	2.49	3.66	2.38
ViSpec 2026.03		1.83	2.52	2.34	2	3.44	2.83	3.01	2.4	3	2.59	3.01	2.68	3	2.58
EAGLE-1 2026.03		1.81	2.41	2.04	1.28	2.46	2.59	2.24	1.93	2.46	2.22	2.33	2.14	2.36	2.11
EAGLE-2 2026.03		1.73	1.79	1.52	1.17	1.7	2.32	1.87	1.95	1.86	2.56	1.78	1.92	1.78	2.02
ViSpec 2026.03		1.67	2.01	1.26	1.12	1.75	2.07	1.57	1.72	1.28	1.52	1.15	1.42	1.29	1.51
Recycling 2026.03		1.6	0.31	0.19	1.87	1.12	1.05	0.94	0.72	0.62	1.29	1	2.37	0.88	1.54
Medusa 2026.03		1.56	1.12	0.77	1.65	0.84	1.32	0.95	1.88	1.07	1.66	1.52	1.06	1.3	1.26
SAM 2026.03		1.33	0.11	0.1	1.23	0.16	1.48	0.51	2.04	0.19	6.53	0.27	2.04	0.23	2.17
Medusa 2026.03		1.29	0.82	0.66	1.01	0.72	1.55	0.92	1.59	0.82	1.99	0.82	1.38	0.8	1.49
PLD 2026.03		1.2	0.14	0.03	1.14	0.01	0.99	1.26	1.13	2.04	1.96	0.48	2.67	0.6	1.77
EAGLE-1 2026.03		1.17	0.89	0.68	1.12	0.73	1.19	0.81	1.21	0.76	1.19	0.77	1.25	0.76	1.22
EAGLE-2 2026.03		1.14	0.65	0.54	1.13	0.57	1.18	0.65	1.22	0.56	1.17	0.58	1.23	0.58	1.2
Lookahead 2026.03		1.09	0.37	0.14	0.97	0.15	0.92	0.52	1.19	1.7	1.91	0.46	1.11	0.52	1.17
SAM 2026.03		1.09	0.31	0.11	0.93	0.07	0.87	0.42	1.14	1.08	1.63	0.34	1.11	0.38	1.13
Lookahead 2026.03		1.06	0.2	0.07	0.77	0.11	1.08	0.43	1.34	0.65	1.47	0.3	0.95	0.33	1.07
EAGLE-3 2026.03		1.02	0.43	0.22	0.83	0.38	1.17	0.22	1.09	0.16	0.85	0.25	0.97	0.24	0.96
Recycling 2026.03		1.02	0.05	0.04	0.8	0.05	1.04	0.17	0.87	0.13	1.08	0.12	1.09	0.11	1.04
PLD 2026.03		1.01	0.03	0	0.78	0	1	0.29	1.04	0.26	1.16	0.2	1.06	0.17	1.05
AR Baseline 2026.03		1	-	-	1	-	1	-	1	-	1	-	1	-	1
AR Baseline 2026.03		1	-	-	1	-	1	-	1	-	1	-	1	-	1
EAGLE-3 2026.03		0.68	0.06	0.07	0.71	0.1	0.73	0.06	0.72	0.01	0.68	0.03	0.69	0.04	0.7