← All reports

Report · 2026-11-10

Qwen3-8B BF16 on 1× H200 (Lium, Jackson) with vLLM 0.33.0

1× NVIDIA H200 Qwen3-8B BF16 weights, BF16 KV cache vLLM 0.33.0 TTFT p99 ≤ 2.0 s · TPOT p99 ≤ 50 ms

Quick-mode measurement on a Lium Secure-tier marketplace node (f092246d, Jackson, United States), 2026-11-10, default vLLM settings, $5.30 per GPU-hour (the host's listed price). Same model, workload and latency target as our other Qwen3-8B reports; the node ran tokenwatt-check first (700 W power limit, its default; 668 TFLOPS BF16; 4.30 TB/s copy bandwidth). $0.37 per million output tokens at that price. Lium provided the credit for this run and checked the node details before publication. Energy is GPU board power.

ConclusionSingle configuration measured under SLO (TTFT p99 ≤ 2.0 s · TPOT p99 ≤ 50 ms).

Results

Each value is the median of 2 runs.

lium-f092246d-qwen3-8b

SLO goodput

output tok/s per node · higher is better
lium-f092246d-qwen3-8b4,014lium-f092246d-qwen3-8b: 4,014 output tok/s per node

Tokens per joule

tok/J, GPU board power · higher is better
lium-f092246d-qwen3-8b5.80lium-f092246d-qwen3-8b: 5.80 tok/J, GPU board power

Decode bandwidth utilization

% of rated HBM bandwidth
lium-f092246d-qwen3-8b40%lium-f092246d-qwen3-8b: 40 % of rated HBM bandwidth

Axis runs to 100% of rated bandwidth.

GPU board power

kW, steady state
lium-f092246d-qwen3-8b0.69lium-f092246d-qwen3-8b: 0.69 kW, steady state

GPU board power only — excludes CPUs, memory, NICs, fans and PSU losses.

All numbers

Cards for equal goodput = 64 × reference goodput ÷ configuration goodput, rounded up.

Metriclium-f092246d-qwen3-8b
Engine buildvLLM 0.33.0 · KV BF16
Decode bandwidth utilization (MBU)40%
GPU board power0.69 kW
SLO goodput4,014 tok/s
Tokens per joule5.80
TTFT p99417 ms
TPOT p9936.9 ms
SLO metyes
Accuracy (gsm8k-5shot)92.0
Accuracy Δ vs. referencereference
Cards for equal goodput64

Setup

Any change to these fields makes it a different report.

Accelerator
1× NVIDIA H200
Rated HBM BW
4.8 TB/s per GPU
Driver / runtime
580.159.03
Host
Lium Secure-tier pod (marketplace host), node f092246d, Jackson, US
Engine
vLLM 0.33.0
Kernel libraries
vllm/vllm-openai:v0.33.0
Model
Qwen3-8B (Dense)
Quantization
BF16 weights, BF16 KV cache
Parallelism
1 replica × TP1
Workload trace
chat-mix-v1
Input tokens
median 1,024 · p90 4,096
Output tokens
median 256 · p90 1,024
Arrival
Poisson, rate swept to SLO boundary
Power source
GPU board power sum (NVIDIA) · 1 Hz · 120 s steady state
Runs per config
2 (variance band ±1.4%)
Accuracy gate
gsm8k-5shot · max drop 1.0 pts

Not covered

Do not extrapolate this report to the following.

  • Whole-node power (no BMC access on this host)
  • Context lengths beyond the chat-mix-v1 trace
  • Quick mode: short windows, few repeats — validation only, not for publishing

Reproduce

Run on the same hardware and software versions. Results should fall within ±1.4%.

shell
tokenwatt run --spec lambda-qwen3-8b-tp1-dp1.spec.yaml --label 'lium-f092246d-qwen3-8b' --role other --out runs/lium-f092246d-qwen3-8b.json

tokenwatt report runs/lium-f092246d-qwen3-8b.json --title 'Qwen3-8B BF16 on 1× H200 (Lium, Jackson) with vLLM 0.33.0'

Download raw JSON