contact

LLM serving that works on the GPUs you actually ship.

We test vLLM and SGLang on RTX PRO 6000 and PCIe multi-GPU servers every release, with RTX 5090 next. We fix what breaks upstream and publish the numbers.

get your stack tested see the results

Same GPUs. Same model. Different stack.

Two runs on rented RTX PRO 6000s. The left lane is the stack as it ships today, the right lane is the same stack after one change. Each lane streams at the speed we measured.

users
as shipped--enforce-eager, prefix caching off
this user-
tunedCUDA graphs + prefix caching on
this user-
$ vllm bench serve --max-concurrency 1 --random-input-len 1024 --random-output-len 512as shipped 17.4 tok/s total tuned 125.0 tok/s total 7.2x

Independent test of a public repo: Puget Systems' Team LLM pack v1.1.0, menu entry 7, run exactly as installed. Puget merged this change as puget-docker-app-packs#12. 2x RTX PRO 6000 Workstation, Nemotron-3-Super-120B-A12B-NVFP4, TP2, vLLM nightly e68efc68. GSM8K: 99/100 shipped vs 98/100 tuned, every reply answered. Each lane streams at the measured output rate per user; the text is illustrative. Full report.

What serves, what breaks.

Every row is one run on rented hardware with a fixed budget. Each one links to its full report: image, flags, commands and cost.

GPUs Model Engine Outcome Number Run
2x RTX PRO 6000 WS cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit vLLM nightly e68efc68 serves Puget pack v1.1.0 entry 1 as installed; the shipped eager flags cost 6.3x to 7.1x at 1 user across two hosts (PR #15, open) 245.6 tok/s at 1 user (34.5 shipped) puget-team-llm-e1
2x RTX PRO 6000 WS cyankiwi/Qwen3.6-27B-AWQ-INT4 vLLM nightly e68efc68 serves Puget pack v1.1.0 entry 2 as installed; the shipped eager flags cost 3.5x at 1 user (PR #15, open) 112.2 tok/s at 1 user (32.4 shipped) puget-team-llm-e2
2x RTX PRO 6000 WS cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit vLLM nightly e68efc68 serves Puget pack v1.1.0 entry 3 as installed; the shipped eager flags cost 6.4x at 1 user (PR #15, open) 221.7 tok/s at 1 user (34.6 shipped) puget-team-llm-e3
2x RTX PRO 6000 WS nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 vLLM nightly e68efc68 serves Puget pack v1.1.0 entry 6 as installed; the shipped eager flags cost 9.1x at 1 user (PR #15, open) 324.3 tok/s at 1 user (35.6 shipped) puget-team-llm-e6
2x RTX PRO 6000 WS nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 vLLM nightly e68efc68 serves Puget pack v1.1.0 entry 7 as installed; the shipped eager flags cost 7.2x at 1 user 125.0 tok/s at 1 user (17.4 shipped) puget-team-llm-e7
2x RTX PRO 6000 WS cyankiwi/gemma-4-26B-A4B-it-AWQ-4bit vLLM nightly e68efc68 serves Puget pack v1.1.0 entry 8 as installed; the shipped eager flags cost 4.8x at 1 user (PR #15, open) 227.3 tok/s at 1 user (47.6 shipped) puget-team-llm-e8
1x RTX PRO 6000 WS nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 vLLM 0.26.0 serves Wowza Video Intelligence Framework's engine config; its --enforce-eager costs 2.8x at 1 user (text only, the video path was not measured) 108.8 tok/s at 1 user (38.9 shipped) wowza-vif
1x RTX 5090 deepseek-ai/DeepSeek-R1-Distill-Llama-8B RunPod worker-vllm, vLLM 0.32.0 serves Hub defaults; the worker's out-of-memory relaunch to eager costs 2.0x at 1 request. MAX_NUM_SEQS=32 keeps graphs and the full context, and boots in 80 s instead of 251 s 97.7 tok/s at 1 request (49.2 after relaunch) runpod-worker-vllm-r1d8b
1x RTX PRO 6000 SE RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic RunPod worker-vllm, vLLM 0.32.0 serves Hub defaults; the same relaunch costs 0-3% on this model. MAX_NUM_SEQS=32 boots in 90 s instead of 210 s 19.2 tok/s at 1 request (19.1 after relaunch) runpod-worker-vllm-l70b
1x RTX PRO 6000 WS nvidia/Qwen3.6-35B-A3B-NVFP4 vLLM 0.28.0 and nightly serves 25 min saturated, graphs on, 0 failed 2.8-2.9K tok/s at 70 users vllm-54331
1x RTX PRO 6000 SE Ornith-1.5-35B-A3B-NVFP4, fp8 KV, video vLLM 0.28.0 serves 25 min saturated with video, 0 failed 2,856 tok/s at 70 users vllm-54331
4x RTX PRO 6000 SE DeepSeek-V4-Flash SGLang v0.5.22 serves 1.10x vLLM on 9 x 90K prompts; the reported gap did not show 8,983 prefill tok/s sglang-33422
4x RTX PRO 6000 SE DeepSeek-V4-Flash vLLM 0.25.1 recipe serves same 9 x 90K prompts 8,171 prefill tok/s sglang-33422-vllm
4x RTX PRO 6000 SE DeepSeek-V4.1-Flash + DSpark vLLM nightly, FlashInfer 0.7.2, vllm#60762 serves 10.7K and 106K prompts with graphs on; crashed on FlashInfer 0.6.18 101.5 decode tok/s vllm-56771
2x RTX PRO 6000 SE nvidia/Qwen3.8-Flash-Next-NVFP4, EP2 + NEXTN SGLang nightly 2026-10-21 (c1c287a4) serves no patches (fix sglang#44480 merged); TP2 still needs #42566, custom all-reduce with expandable_segments still needs #44448 187.5 / 820.3 / 1,477.8 tok/s at 1 / 16 / 64 users, GSM8K 0.995 sglang-44291-stack
1x RTX PRO 6000 SE Apertus-70B FP8 vLLM 0.25.1 serves the reported garbage output did not reproduce GSM8K-200 0.775 vllm-61100
1x RTX PRO 6000 SE DeepSeek-V2-Lite-Chat-FP8 vLLM 0.31.0 crashes at startup when routed to CUTLASS; the default config is fine - vllm-47990
1x RTX PRO 6000 nvidia/Qwen3.6-35B-A3B-NVFP4 vLLM, FlashInfer 0.7.0.post1 fixed CUDA-graph crash at startup, gone with FlashInfer 0.7.1 - vllm-49476

18 runs. Updated 2026-10-25. All kits: github.com/adi-suresh01/sm120-kit

How a run works.

  1. 01

    Real hardware

    Rented RTX PRO 6000 Workstation and Server Edition boxes, one card or several over PCIe, the same parts integrators ship. One RTX 5090 run so far; more are next.

  2. 02

    A fixed budget

    Every run gets a GPU budget before it starts, and the pod shuts down when the run ends.

  3. 03

    A reproducible report

    Image digest, flags, driver, commands and cost. Rerun it on your own box and you should get the same numbers.

  4. 04

    Fixes go upstream

    When vLLM, SGLang or FlashInfer is the problem, the fix goes to the project as a PR, not into a fork you have to maintain.

Start free. Pay when it's useful.

Pilot Free

Your current pack on one GPU config, tested in public. You get the report, and any fixes as PRs.

start here
Acceptance test from $500 per deployment

Your customer's exact model and prompt lengths on their box, before it ships. White-labeled report.

Fix sprint from $2.5K fixed

One broken deployment, fixed to a target we agree on up front.

Stack Care from $1.5K / month

Up to 3 GPU configs x 5 models, re-tested on every vLLM, SGLang and FlashInfer release. Known-good image tags and flags, 2-business-day escalations, fixes upstream.

for integrators

Every one starts with the same email: get your stack tested.

I'm Aditya, an ML engineer who most recently served models on PCIe RTX PRO 6000 clusters at Kos AI, now founder of Provedex.

Every number on this page comes from sm120-kit, the public harness that rents the GPUs, runs the matrix and writes the report. More on GitHub.

Get your stack tested.

Tell us the GPUs, model and engine, and what breaks or feels slow.

email hello@provedex.io opens a short template