← All reports

Report · 2026-11-10

Qwen3-8B BF16 on 1× RTX PRO 6000 Blackwell Server Edition (Lium, Almaty) with vLLM 0.33.0

1× NVIDIA RTX PRO 6000 Blackwell Server Edition Qwen3-8B BF16 weights, BF16 KV cache vLLM 0.33.0 TTFT p99 ≤ 2.0 s · TPOT p99 ≤ 50 ms

Quick-mode measurement on a Lium Secure-tier marketplace node (6b40f6ba, Almaty, Kazakhstan), 2026-11-10, default vLLM settings, $1.75 per GPU-hour (the host's listed price). Same model, workload and latency target as our other Qwen3-8B reports; the node ran tokenwatt-check first (600 W power limit, its default; 406 TFLOPS BF16; 1.46 TB/s copy bandwidth). $0.39 per million output tokens at that price. Lium provided the credit for this run and checked the node details before publication. Energy is GPU board power.

ConclusionSingle configuration measured under SLO (TTFT p99 ≤ 2.0 s · TPOT p99 ≤ 50 ms).

Results

Each value is the median of 2 runs.

lium-6b40f6ba-qwen3-8b

SLO goodput

output tok/s per node · higher is better
lium-6b40f6ba-qwen3-8b1,234lium-6b40f6ba-qwen3-8b: 1,234 output tok/s per node

Tokens per joule

tok/J, GPU board power · higher is better
lium-6b40f6ba-qwen3-8b2.66lium-6b40f6ba-qwen3-8b: 2.66 tok/J, GPU board power

Decode bandwidth utilization

% of rated HBM bandwidth

Not measured in this report.

GPU board power

kW, steady state
lium-6b40f6ba-qwen3-8b0.46lium-6b40f6ba-qwen3-8b: 0.46 kW, steady state

GPU board power only — excludes CPUs, memory, NICs, fans and PSU losses.

All numbers

Cards for equal goodput = 64 × reference goodput ÷ configuration goodput, rounded up.

Metriclium-6b40f6ba-qwen3-8b
Engine buildvLLM 0.33.0 · KV BF16
Decode bandwidth utilization (MBU)—
GPU board power0.46 kW
SLO goodput1,234 tok/s
Tokens per joule2.66
TTFT p99666 ms
TPOT p9945.1 ms
SLO metyes
Accuracy (gsm8k-5shot)92.0
Accuracy Δ vs. referencereference
Cards for equal goodput64

Setup

Any change to these fields makes it a different report.

Accelerator
1× NVIDIA RTX PRO 6000 Blackwell Server Edition
Rated HBM BW
—
Driver / runtime
595.91.07
Host
Lium Secure-tier pod (marketplace host), node 6b40f6ba, Almaty, KZ
Engine
vLLM 0.33.0
Kernel libraries
vllm/vllm-openai:v0.33.0
Model
Qwen3-8B (Dense)
Quantization
BF16 weights, BF16 KV cache
Parallelism
1 replica × TP1
Workload trace
chat-mix-v1
Input tokens
median 1,024 · p90 4,096
Output tokens
median 256 · p90 1,024
Arrival
Poisson, rate swept to SLO boundary
Power source
GPU board power sum (NVIDIA) · 1 Hz · 120 s steady state
Runs per config
2 (variance band ±1.3%)
Accuracy gate
gsm8k-5shot · max drop 1.0 pts

Not covered

Do not extrapolate this report to the following.

  • Whole-node power (no BMC access on this host)
  • Context lengths beyond the chat-mix-v1 trace
  • Quick mode: short windows, few repeats — validation only, not for publishing

Reproduce

Run on the same hardware and software versions. Results should fall within ±1.3%.

shell
tokenwatt run --spec lambda-qwen3-8b-tp1-dp1.spec.yaml --label 'lium-6b40f6ba-qwen3-8b' --role other --out runs/lium-6b40f6ba-qwen3-8b.json

tokenwatt report runs/lium-6b40f6ba-qwen3-8b.json --title 'Qwen3-8B BF16 on 1× RTX PRO 6000 Blackwell Server Edition (Lium, Almaty) with vLLM 0.33.0'

Download raw JSON