Qwen3-Coder-30B-A3B FP8 on 1× H100 SXM (Lium spot): vLLM defaults vs DeepGEMM
Quick-mode measurement on one Lium spot-tier H100 SXM (Atlanta), 2026-11-13/14, $1.50 per GPU-hour at rent time, vllm/vllm-openai:v0.33.0, both with --enable-auto-tool-choice --tool-call-parser qwen3_coder. VLLM_USE_DEEP_GEMM=1 (vLLM's Qwen3-Coder recipe for Hopper) switched the MoE layers to DeepGEMM and didn't move the capacity: 4,600 vs 4,610 tok/s at the same 11.31 req/s; failing steps were queueing at saturation. $0.09 per million output tokens at $1.50/h. Lium has given TokenWatt credits for earlier node evaluations and this run was paid from that balance; Lium had no input on it. Write-up: /blog/mi350x-vs-h100-software-and-price.html
Results
Each value is the median of 2 runs.
SLO goodput
Tokens per joule
Decode bandwidth utilization
Axis runs to 100% of rated bandwidth.
GPU board power
GPU board power only — excludes CPUs, memory, NICs, fans and PSU losses.
All numbers
Cards for equal goodput = 64 × reference goodput ÷ configuration goodput, rounded up.
| Metric | vLLM defaults | DeepGEMM FP8 MoE |
|---|---|---|
| Engine build | vLLM 0.33.0 · KV BF16 | vLLM 0.33.0 · KV BF16 |
| Decode bandwidth utilization (MBU) | 59% | 56% |
| GPU board power | 0.60 kW | 0.58 kW |
| SLO goodput | 4,610 tok/s | 4,600 tok/s |
| Tokens per joule | 7.62 | 7.88 |
| TTFT p99 | 332 ms | 288 ms |
| TPOT p99 | 41.7 ms | 43.4 ms |
| SLO met | yes | yes |
| Accuracy (gsm8k-5shot) | 94.0 | 96.0 |
| Accuracy Δ vs. reference | reference | +2.0 pts |
| Cards for equal goodput | 64 | 65 |
Setup
Any change to these fields makes it a different report.
- Accelerator
- 1× NVIDIA H100 80GB HBM3
- Rated HBM BW
- 3.35 TB/s per GPU
- Driver / runtime
- 580.105.08
- Host
- Lium spot-tier pod (interruptible marketplace host), node 00750dfa, Atlanta, US; $1.50 per GPU-hour on 2026-11-13
- Engine
- vLLM 0.33.0
- Kernel libraries
- vllm/vllm-openai:v0.33.0 --enable-auto-tool-choice --tool-call-parser qwen3_coder
- Model
- Qwen3-Coder-30B-A3B-Instruct-FP8 (MoE)
- Quantization
- FP8 weights, BF16 KV cache
- Parallelism
- 1 replica × TP1
- Workload trace
- chat-mix-v1
- Input tokens
- median 1,024 · p90 4,096
- Output tokens
- median 256 · p90 1,024
- Arrival
- Poisson, rate swept to SLO boundary
- Power source
- GPU board power sum (NVIDIA) · 1 Hz · 120 s steady state
- Runs per config
- 2 (variance band ±0.7%)
- Accuracy gate
- gsm8k-5shot · max drop 1.0 pts
Not covered
Do not extrapolate this report to the following.
- Whole-node power (no BMC access on this host)
- Context lengths beyond the chat-mix-v1 trace
- Quick mode: short windows, few repeats — validation only, not for publishing
- Accuracy: quick mode scores GSM8K on 50 questions with one seed, so one question is 2 points and the 1-point gate can't tell a real drop from sampling noise. A −4 pt flag is two questions
- Spot (interruptible) node; it wasn't interrupted during the runs
Reproduce
Run on the same hardware and software versions. Results should fall within ±0.7%.
Raw run files, logs and the exact launch commands: bench/results/h100-tuned-2026-11/ in the TokenWatt repository (README.md). tokenwatt run --spec lambda-qwen3-coder-30b-a3b-instruct-fp8-tp1-dp1.spec.yaml --label 'vLLM defaults' --role baseline --out runs/vllm-defaults.json tokenwatt run --spec lambda-qwen3-coder-30b-a3b-instruct-fp8-tp1-dp1.spec.yaml --label 'DeepGEMM FP8 MoE' --role tuned --out runs/deepgemm-fp8-moe.json tokenwatt report runs/vllm-defaults.json runs/deepgemm-fp8-moe.json --title 'Qwen3-Coder-30B-A3B FP8 on 1× H100 SXM (Lium spot): vLLM defaults vs DeepGEMM'