Public reports
Results, with the receipts attached.
Each report pins hardware, software, model, workload and SLO, lists what it doesn't cover, and ships with raw JSON and the command that reproduces it.
2026-11-21
harness-hub's pi adapter (LoRA) on Qwen3.8-27B FP8, vLLM 0.33.0: Terminal-Bench 2.1 subset under pi, cost and stability on a shared H200
3.0×fewer turns per solved task, pi adapter vs base model, FP8
2026-11-20
Prefix-cache hits on real coding-agent traffic: Qwen3.8-27B and Qwen3.6-35B-A3B on one H200, vLLM 0.33.0
94.3%of Qwen3.8-27B prompt tokens from cache, real agent traffic
2026-11-20
MTP speculative decoding on Qwen3.8-27B FP8, 1× H100 NVL, vLLM 0.33.0: 0 to 4 draft tokens
2.65×single-request decode, 3 draft tokens vs MTP off
2026-11-20
Qwen3.8-27B and Qwen3.6-35B-A3B on our API, 175 single-file Python tasks: thinking on vs off
26×Qwen3.8-27B cost, thinking on vs off, for +5.7 points
2026-11-18
Qwen3-8B BF16 on 1× AMD Instinct MI350X: vLLM's ROCm image vs AMD's image and settings
2.36×SLO goodput, AMD image + AMD settings
vs. vLLM ROCm image, defaults
vs. vLLM ROCm image, defaults
2026-11-18
Qwen3-Coder-30B-A3B FP8 on 1× AMD Instinct MI350X: vLLM's ROCm image vs AMD's image and settings
2026-11-18
Qwen3-8B BF16 on 1× H100 SXM (Lium spot): vLLM defaults vs vLLM's recommended setting
2026-11-18
Qwen3-Coder-30B-A3B FP8 on 1× H100 SXM (Lium spot): vLLM defaults vs DeepGEMM
1.00×SLO goodput, DeepGEMM FP8 MoE
vs. vLLM defaults
vs. vLLM defaults
2026-11-13
Prefix caching on a hybrid model, one H100: Qwen3.8-27B FP8, vLLM 0.31.0, chat-like traffic
1.67×SLO goodput, 90% shared prefix (55.6% cache hits)
vs. No prompt reuse (0% cache hits)
vs. No prompt reuse (0% cache hits)
2026-11-13
Prefix caching on a hybrid model, one H100: Qwen3.8-27B FP8, vLLM 0.31.0, 4,000-token prompts
3.20×SLO goodput, 90% shared prefix (74.2% cache hits)
vs. No prompt reuse (0% cache hits)
vs. No prompt reuse (0% cache hits)
2026-11-10
Qwen3-8B BF16 on 1× B200 (Lium, Kansas City) with vLLM 0.33.0
2026-11-10
Qwen3-8B BF16 on 1× H100 80GB HBM3 (Lium, Helsinki) with vLLM 0.33.0
2026-11-10
Qwen3-8B BF16 on 1× H200 (Lium, Jackson) with vLLM 0.33.0
2026-11-10
Qwen3-8B BF16 on 1× RTX PRO 6000 Blackwell Server Edition (Lium, Almaty) with vLLM 0.33.0
2026-11-10
Qwen3-8B BF16 on 1× RTX PRO 6000 Blackwell Server Edition (Lium, Dublin) with vLLM 0.33.0
2026-11-02
Long-context capacity on one H100: ~8K input tokens, Qwen3-30B-A3B-Instruct-2507 FP8, vLLM 0.31.0
2026-11-02
Long-context capacity on one H100: ~16K input tokens, Qwen3-30B-A3B-Instruct-2507 FP8, vLLM 0.31.0
2026-10-29
Prefix caching on one H100: Qwen3-30B-A3B-Instruct-2507 FP8, vLLM 0.31.0
1.79×SLO goodput, 90% shared prefix (89.2% cache hits)
vs. No prompt reuse (0% cache hits)
vs. No prompt reuse (0% cache hits)
2026-10-27
DeepSeek-V4.1-Flash on 4× H200 with vLLM 0.32.0 (TP4, FP8 KV cache)
2026-10-22
Qwen3-8B BF16 on 1× RTX PRO 6000 Blackwell Server Edition with vLLM 0.31.0
2026-10-13
Qwen3-8B BF16 on 1× H100 PCIe 80GB with vLLM 0.31.0
2026-10-13
Qwen3-8B BF16 on 1× H100 NVL 94GB with vLLM 0.31.0
2026-10-13
Qwen3-8B BF16 on 1× H100 SXM 80GB with vLLM 0.31.0
2026-10-11
Qwen3-8B BF16 on 1× A100 SXM 80GB with vLLM 0.31.0
2026-10-11
Qwen3-8B BF16 on 1× A40 with vLLM 0.31.0
2026-10-11
Qwen3-8B BF16 on 1× L40S with vLLM 0.31.0
2026-10-11
Qwen3-8B FP8 on 1× L40S with vLLM 0.31.0
2026-10-11
Qwen3-8B BF16 on 1× RTX 4090 with vLLM 0.31.0
2026-10-11
Qwen3-8B FP8 on 1× RTX 4090 with vLLM 0.31.0
2026-10-10
vLLM 0.31.0 vs SGLang 0.5.21 on 1× H100 SXM, Qwen3-30B-A3B FP8
1.01×SLO goodput, SGLang 0.5.21
vs. vLLM 0.31.0
vs. vLLM 0.31.0
2026-10-09
Llama-3.3-70B FP8 on 1× H200: prefill chunk size (max-num-batched-tokens)
2026-10-09
Qwen3-30B-A3B FP8 on 1× H200 with vLLM 0.31.0
2026-10-08
Tuned vs default fused-MoE kernels: Qwen3-30B-A3B FP8 on 1× H100 SXM
1.00×SLO goodput, default kernels (repeat)
vs. default kernels
vs. default kernels
2026-10-06
Qwen3-30B-A3B FP8 on 1× H100 SXM with vLLM 0.31.0
2026-10-06Draft
Qwen3-30B-A3B FP8 on 1× H100: BF16 vs FP8 KV cache
2026-10-06Example data
Mid-size MoE decode on an 8-GPU node
1.36×SLO goodput, Tuned
vs. Vendor latest
vs. Vendor latest