Our new X account is live! Follow @wizwand_team for updates

Large Language Model Throughput on ShareGPT v3

8.85Throughput (req/s)

vLLM-RA

Updated 4d ago

Evaluation Results

Method	Links
vLLM-RA 2024.02		8.85
vLLM-RA 2024.02		8.76
vLLM-PS 2024.02		6.29
vLLM-RA 2024.02		5.9
vLLM-RA 2024.02		5.85
vLLM-PS 2024.02		5.4
vLLM 2024.02		5.03
vLLM-PS 2024.02		4.92
vLLM-PS 2024.02		4.82
vLLM 2024.02		3.68
vLLM-RA 2024.02		3.65
vLLM-RA 2024.02		3.64
vLLM 2024.02		3.54
vLLM-PS 2024.02		3.01
vLLM 2024.02		2.6
vLLM-PS 2024.02		2.55
vLLM 2024.02		2.15
vLLM-RA 2024.02		1.71
vLLM-RA 2024.02		1.69
vLLM 2024.02		1.52
vLLM-PS 2024.02		1.44
vLLM-PS 2024.02		1.23
vLLM 2024.02		0.99
vLLM 2024.02		0.66