Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

LLM Inference on LLaMA2 7B

11.09TTFT (ms)

JIT+CUDA

8.013628.779349.54570.3107Apr 25, 2026
Updated 1mo ago

Evaluation Results

MethodLinks
2026.04
11.09---
2026.04
11.17---
2026.04
12.7---
2026.04
12.95---
2026.04
13.36---
2026.04
14.91---
2026.04
15.17---
2026.04
15.53---
2026.04
16---
2026.04
17.02---
2026.04
17.47---
2026.04
17.79---
2026.04
22---
2026.04
24.65---
2026.04
28---
2026.04
29---
2026.04
29.75---
2026.04
31.97---
2026.04
33.66---
2026.04
34.01---
2026.04
35.54---
2026.04
41.03---
2026.04
43.27---
2026.04
46.18---
2026.04
46.47---
2026.04
48---
2026.04
48.96---
2026.04
68---
2026.04
68---
2026.04
69---
2026.04
86---
2026.04
86---
2026.04
88---
2026.03
--1,052.24-
2026.03
-3.046,075.9477.4
2026.03
-501,937.3884.1
2026.03
-0.311,1489.1