Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Decoding Throughput on Llama 2 70B v1.0 (inference)

23.5Throughput (TOK/s)

QTIP

8.191212.165616.1420.1144Jun 17, 2024
Updated 4d ago

Evaluation Results

MethodLinks
2024.06
23.5
2024.06
22.2
2024.06
19.1
2024.06
16.3
2024.06
8.78