Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Language Model Inference on Sequence Bucket Medium
Loading...
52.88
Latency (ms)
COREY
52.2404
56.5577
60.875
65.1923
Apr 12, 2026
Latency (ms)
Throughput (k tok/s)
DRAM Memory (B/token)
Updated 4d ago
Evaluation Results
Method
Method
Links
Latency (ms)
Throughput (k tok/s)
DRAM Memory (B/token)
COREY
Precision=FP16
2026.04
52.88
114.1
717
Static Fusion
Precision=FP16
2026.04
56.75
106.3
778
No Fusion
Precision=FP16
2026.04
68.87
87.6
940
Feedback
Search any
task
Search any
task