Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Inference Latency on A100 GPU

25.4Latency (ms)

AWQ

22.76840.53458.376.066Jun 10, 2024
Updated 1mo ago

Evaluation Results

MethodLinks
2024.06
25.4
2024.06
25.4
2024.06
26.7
2024.06
27.2
2024.06
27.4
2024.06
27.5
2024.06
27.7
2024.06
28.5
2024.06
28.5
2024.06
30.1
2024.06
31.1
2024.06
31.4
2024.06
31.7
2024.06
31.7
2024.06
31.8
2024.06
31.8
2024.06
31.9
2024.06
32.6
2024.06
33.3
2024.06
33.8
2024.06
33.9
2024.06
34.2
2024.06
34.3
2024.06
34.7
2024.06
36.2
2024.06
36.8
2024.06
38.8
2024.06
38.8
2024.06
42.2
2024.06
43.1
2024.06
67.7
2024.06
67.7
2024.06
68
2024.06
68
2024.06
70.9
2024.06
71
2024.06
72.1
2024.06
72.6
2024.06
72.9
2024.06
75.1
2024.06
76.4
2024.06
77.2
2024.06
77.5
2024.06
78.3
2024.06
81.9
2024.06
82.5
2024.06
90.7
2024.06
91.2