← All reports

Report · 2026-11-10

Qwen3-8B BF16 on 1× H100 80GB HBM3 (Lium, Helsinki) with vLLM 0.33.0

1× NVIDIA H100 80GB HBM3 Qwen3-8B BF16 weights, BF16 KV cache vLLM 0.33.0 TTFT p99 ≤ 2.0 s · TPOT p99 ≤ 50 ms

Quick-mode measurement on a Lium Secure-tier marketplace node (df4234ab, Helsinki, Finland), 2026-11-10, default vLLM settings, $2.50 per GPU-hour (the host's listed price). Same model, workload and latency target as our other Qwen3-8B reports; the node ran tokenwatt-check first (700 W power limit, its default; 684 TFLOPS BF16; 3.05 TB/s copy bandwidth). $0.19 per million output tokens at that price. Lium provided the credit for this run and checked the node details before publication. Energy is GPU board power.

ConclusionSingle configuration measured under SLO (TTFT p99 ≤ 2.0 s · TPOT p99 ≤ 50 ms).

Results

Each value is the median of 2 runs.

lium-df4234ab-qwen3-8b

SLO goodput

output tok/s per node · higher is better
lium-df4234ab-qwen3-8b3,580lium-df4234ab-qwen3-8b: 3,580 output tok/s per node

Tokens per joule

tok/J, GPU board power · higher is better
lium-df4234ab-qwen3-8b5.30lium-df4234ab-qwen3-8b: 5.30 tok/J, GPU board power

Decode bandwidth utilization

% of rated HBM bandwidth
lium-df4234ab-qwen3-8b51%lium-df4234ab-qwen3-8b: 51 % of rated HBM bandwidth

Axis runs to 100% of rated bandwidth.

GPU board power

kW, steady state
lium-df4234ab-qwen3-8b0.68lium-df4234ab-qwen3-8b: 0.68 kW, steady state

GPU board power only — excludes CPUs, memory, NICs, fans and PSU losses.

All numbers

Cards for equal goodput = 64 × reference goodput ÷ configuration goodput, rounded up.

Metriclium-df4234ab-qwen3-8b
Engine buildvLLM 0.33.0 · KV BF16
Decode bandwidth utilization (MBU)51%
GPU board power0.68 kW
SLO goodput3,580 tok/s
Tokens per joule5.30
TTFT p99403 ms
TPOT p9937.2 ms
SLO metyes
Accuracy (gsm8k-5shot)92.0
Accuracy Δ vs. referencereference
Cards for equal goodput64

Setup

Any change to these fields makes it a different report.

Accelerator
1× NVIDIA H100 80GB HBM3
Rated HBM BW
3.35 TB/s per GPU
Driver / runtime
580.173.02
Host
Lium Secure-tier pod (marketplace host), node df4234ab, Helsinki, FI
Engine
vLLM 0.33.0
Kernel libraries
vllm/vllm-openai:v0.33.0
Model
Qwen3-8B (Dense)
Quantization
BF16 weights, BF16 KV cache
Parallelism
1 replica × TP1
Workload trace
chat-mix-v1
Input tokens
median 1,024 · p90 4,096
Output tokens
median 256 · p90 1,024
Arrival
Poisson, rate swept to SLO boundary
Power source
GPU board power sum (NVIDIA) · 1 Hz · 120 s steady state
Runs per config
2 (variance band ±0.4%)
Accuracy gate
gsm8k-5shot · max drop 1.0 pts

Not covered

Do not extrapolate this report to the following.

  • Whole-node power (no BMC access on this host)
  • Context lengths beyond the chat-mix-v1 trace
  • Quick mode: short windows, few repeats — validation only, not for publishing

Reproduce

Run on the same hardware and software versions. Results should fall within ±0.4%.

shell
tokenwatt run --spec lambda-qwen3-8b-tp1-dp1.spec.yaml --label 'lium-df4234ab-qwen3-8b' --role other --out runs/lium-df4234ab-qwen3-8b.json

tokenwatt report runs/lium-df4234ab-qwen3-8b.json --title 'Qwen3-8B BF16 on 1× H100 80GB HBM3 (Lium, Helsinki) with vLLM 0.33.0'

Download raw JSON