Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Inference Latency on A100 GPU

25.4Latency (ms)

AWQ

22.76840.53458.376.066Jun 10, 2024
Updated 1mo ago

Evaluation Results

MethodLinks
2024.06
25.4-
2024.06
25.4-
2024.06
26.7-
2024.06
27.2-
2024.06
27.4-
2024.06
27.5-
2024.06
27.7-
2024.06
28.5-
2024.06
28.5-
2024.06
30.1-
2024.06
31.1-
2024.06
31.4-
2024.06
31.7-
2024.06
31.7-
2024.06
31.8-
2024.06
31.8-
2024.06
31.9-
2024.06
32.6-
2024.06
33.3-
2024.06
33.8-
2024.06
33.9-
2024.06
34.2-
2024.06
34.3-
2024.06
34.7-
2024.06
36.2-
2024.06
36.8-
2024.06
38.8-
2024.06
38.8-
2024.06
42.2-
2024.06
43.1-
2024.06
67.7-
2024.06
67.7-
2024.06
68-
2024.06
68-
2024.06
70.9-
2024.06
71-
2024.06
72.1-
2024.06
72.6-
2024.06
72.9-
2024.06
75.1-
2024.06
76.4-
2024.06
77.2-
2024.06
77.5-
2024.06
78.3-
2024.06
81.9-
2024.06
82.5-
2024.06
90.7-
2024.06
91.2-
2026.06
-6.5
2026.06
-9
2026.06
-0.7
2026.06
-4.9