Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Training Memory Usage Profiling on Qwen3-32B 16×H100s
Loading...
38.94
Memory Footprint (Seq 128K)
FPDT
38.6652
40.5201
42.375
44.2299
Feb 24, 2026
Memory Footprint (Seq 128K)
Memory Footprint (Seq 256K)
Memory Footprint (Seq 512K)
Memory Footprint (Seq 1M)
Memory Footprint (Seq 2M)
Memory Footprint (Seq 3M)
Memory Footprint (Seq 4M)
Memory Footprint (Seq 5M)
Updated 4d ago
Evaluation Results
Method
Method
Links
Memory Footprint (Seq 128K)
Memory Footprint (Seq 256K)
Memory Footprint (Seq 512K)
Memory Footprint (Seq 1M)
Memory Footprint (Seq 2M)
Memory Footprint (Seq 3M)
Memory Footprint (Seq 4M)
Memory Footprint (Seq 5M)
FPDT
Context Parallelism Sc...
2026.02
38.94
39.47
40.54
42.66
46.91
52.27
57.77
-
UPipe
Context Parallelism Sc...
2026.02
39.98
40.84
42.72
46.84
55.65
64.47
73.28
-
Ulysses
Context Parallelism Sc...
2026.02
40.13
41.16
44.1
50.27
62.6
-
-
-
Ring
Context Parallelism Sc...
2026.02
40.14
41.16
44.22
50.51
63.11
-
-
-
Native-PyTorch
Context Parallelism Sc...
2026.02
45.81
53.69
69.47
-
-
-
-
-
Feedback
Search any
task
Search any
task