Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Language Model Inference on Sequence Bucket Short
Loading...
39.26
Latency (ms)
COREY
38.748
42.204
45.66
49.116
Apr 12, 2026
Latency (ms)
Throughput (k tok/s)
DRAM Usage (Bytes/token)
Updated 4d ago
Evaluation Results
Method
Method
Links
Latency (ms)
Throughput (k tok/s)
DRAM Usage (Bytes/token)
COREY
Precision=FP16
2026.04
39.26
38.3
2,120
Static Fusion
Precision=FP16
2026.04
42.61
35.3
2,319
No Fusion
Precision=FP16
2026.04
52.06
28.9
2,805
Feedback
Search any
task
Search any
task