Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Inference Latency on OPT model family

6.2Latency (ms)

AWQ

4.28417.21730.1543.083Jun 10, 2024
Updated 1mo ago

Evaluation Results

MethodLinks
2024.06
6.2
2024.06
6.2
2024.06
6.2
2024.06
6.3
2024.06
6.3
2024.06
6.3
2024.06
6.4
2024.06
7.8
2024.06
8.2
2024.06
8.3
2024.06
11.7
2024.06
11.8
2024.06
11.8
2024.06
12.1
2024.06
12.1
2024.06
12.3
2024.06
12.3
2024.06
12.3
2024.06
12.4
2024.06
12.5
2024.06
12.6
2024.06
12.6
2024.06
12.6
2024.06
13.3
2024.06
15
2024.06
15.1
2024.06
15.5
2024.06
15.7
2024.06
15.7
2024.06
15.9
2024.06
15.9
2024.06
16.1
2024.06
16.3
2024.06
16.3
2024.06
16.3
2024.06
16.3
2024.06
16.6
2024.06
16.7
2024.06
16.7
2024.06
16.8
2024.06
16.9
2024.06
16.9
2024.06
16.9
2024.06
17
2024.06
19.5
2024.06
19.7
2024.06
19.8
2024.06
19.9
2024.06
20
2024.06
20
2024.06
20.8
2024.06
20.9
2024.06
20.9
2024.06
21
2024.06
21.1
2024.06
21.5
2024.06
22.2
2024.06
23.6
2024.06
23.7
2024.06
24.5
2024.06
24.5
2024.06
24.7
2024.06
25.4
2024.06
26.4
2024.06
27.1
2024.06
29.5
2024.06
30.1
2024.06
30.7
2024.06
31.5
2024.06
33.2
2024.06
39.5
2024.06
40.9
2024.06
40.9
2024.06
42.9
2024.06
45.7
2024.06
50.4
2024.06
51.5
2024.06
51.7
2024.06
54.1