Share your thoughts, 1 month free Claude Pro on usSee more

LLM Inference Scheduling on Alpaca

0.52Average Per-token Latency (s/token)

TIE

Updated 17d ago

Evaluation Results

Method	Links
TIE 2026.04		0.52	0.93	54.18	123.02
SSJF 2026.04		0.71	1.47	62.93	140.8
LTR 2026.04		0.83	1.76	65.33	142.17
FCFS 2026.04		1.45	3.62	83.65	159.72
TIE 2026.04		1.54	2.81	146.14	359.08
SSJF 2026.04		2.06	4.32	171.14	395.78
LTR 2026.04		2.36	5.14	174.31	394.1
FCFS 2026.04		4.52	11.36	235.64	444.52