Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Decoding Throughput on Llama 2 7B inference v1.0

188Decoding Throughput (TOK/s)

QTIP

50.61686.283121.95157.617Jun 17, 2024
Updated 4d ago

Evaluation Results

MethodLinks
2024.06
188
2024.06
186
2024.06
161
2024.06
140
2024.06
81.5
2024.06
55.9