Qwen3
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Runtime Speed | Qwen3 Query Projection Module | Throughput (k tokens/sec)92.57 | 90 | |
| Language Modeling | Qwen3 (val) | Validation Loss2.493 | 49 | |
| Training Efficiency Profiling | Qwen3 profiling S=2048 (train) | Per-Step Latency (ms)104.1 | 40 | |
| Training Efficiency Profiling | Qwen3 profiling S=4096 (train) | Per-step Latency (ms)108 | 39 | |
| Large Language Model Evaluation | Qwen3-0.6B Average (test) | Average Performance47.83 | 38 | |
| Training Efficiency Profiling | Qwen3 (train) | Peak GPU Memory3.1 | 31 | |
| Training Iteration Efficiency | Qwen3-0.6B base (train) | Per-step Latency (ms)100.2 | 18 | |
| Output Equivalence | Qwen3 | Exact Match65.6 | 13 | |
| LLM Inference | Qwen3 Samsung Galaxy S25 Ultra 0.6B (test) | Prefill Throughput (min)1,709.9 | 12 | |
| Long-Context Generation | Qwen3 Context length (60K) | Throughput Speedup (α)5.89 | 12 | |
| Long-Context Generation | Qwen3 Context length 40K | Throughput Speedup (α)5.37 | 12 | |
| Long-Context Generation | Qwen3 Context length 30K | Throughput Speedup (α)4.37 | 12 | |
| Long-Context Generation | Qwen3 Context length 20K | Throughput Speedup (α)3.73 | 12 | |
| LLM Inference | Qwen3 Google Pixel 9 Pro XL 0.6B (test) | Prefill Throughput (min, tokens/sec)591.01 | 10 | |
| LLM Training Step Performance | Qwen3-1.7B S=2048 (base variant) | Per-Step Latency (ms)113.8 | 9 | |
| Training Iteration Efficiency | Qwen3-8B base (train) | Per-Step Latency (ms)131.6 | 9 | |
| Training Iteration Efficiency | Qwen3-1.7B base (train) | Per-Step Latency (ms)101.7 | 9 | |
| LLM Training Step Performance | Qwen3-8B S=512 base variant | Per-Step Latency (ms)140.8 | 8 | |
| Data Selection | QWEN3-4B | Wall-clock Time1 | 8 | |
| Training Iteration Efficiency | Qwen3-14B base (train) | Per-step Latency (ms)150.1 | 7 | |
| Training Throughput | Qwen3-30B-A3B workload | Throughput (tokens/s)280,000 | 7 | |
| Kernel-level Attention Speed and Memory Analysis | Qwen3-8B model dimensions (H=32, Hk=8, d=128, GQA 4:1) on A100 GPU (test) | Forward Pass Time (ms)27.1 | 7 | |
| Decode Throughput | Qwen3 0.6B | Throughput (tok/s)464.5 | 6 | |
| Reranker Attack | Qwen3 reranker 0.6B | SR@594.17 | 6 | |
| Watermark Removal | Qwen3 8B | DIPMark14.21 | 6 |
Showing 25 of 37 rows