Research, measured

The real cost of a self-hosted 96GB LLM lane.

Not a spreadsheet of estimates. One machine, one cutover, one week of production traffic: board power, sustained throughput, and the electricity bill per 1M output tokens, set against the metered API alternative. The numbers below are the receipts; the method is in the full field guide.

What was measured, and on what

One RTX PRO 6000 Blackwell Max-Q (96GB GDDR7, 300 W TBP) in a compact build, serving Qwen3.8-27B with a 262,144-token native window. Two lanes: a vLLM 0.24 BF16 baseline and an SGLang NVFP4 lane with DFlash2 speculative decoding. Run 21 August 2026; the full configs, the 137-sample thermal log, and the retrieval test are inthe Blackwell field guide.

Metric	Measured	Condition
Serial decode, 1,024-token output	141.49 tok/s	median end-to-end; 44.6 ms median TTFT
Aggregate, concurrency 2	281.31 tok/s	p50 TTFT 204.7 ms
Aggregate, concurrency 4	408.62 tok/s	configured cap; p50 TTFT 83.8 ms
Full native context	261,494 prompt tokens	99.75% of the 262,144 window; TTFT 99.95 s; planted needle retrieved correctly
GPU board power at saturation	327.61 W peak	137 one-second samples; 98.26% mean utilization; 91°C peak

The electricity bill, per 1M output tokens

The energy anchor is the measured board power at saturation: 300 W sustained (the card's TBP), with a ~100 W system overhead for CPU, cooling and the rest of the compact build treated as an operator estimate, not a measurement. Total: 0.40 kWh per hour of saturation. The lane ran 12 hours per day in this workload, priced at $0.135/kWh (INR 11/kWh residential).

Scenario	Output tokens / day (12 h)	kWh per 1M tokens	Electricity per 1M tokens	Daily bill (12 h)
Serial (1 concurrent request)agent chat, one user at a time	6.1M	0.79 kWh	$0.106	$0.65 (53 INR)
Saturated (concurrency 4)agent fleet, four jobs at once	17.7M	0.27 kWh	$0.037	$0.65 (53 INR)

Alternative	Output cost per 1M tokens	What else you pay
Metered API (US frontier-class, August 2026)	$1.25 to $3.00	Per-token meter, rate limits, context ceiling, and the data leaving the machine
Self-hosted lane, electricity only	$0.11 to $0.04	Hardware, cooling, and your time keeping it warm

published list prices for US frontier-class models, output tokens, August 2026. Check current rates before quoting.

Where the lane actually wins

My take: the electricity math is not the argument.

At 0.11 to 0.04 dollars per 1M output tokens, the lane undercuts a metered API by a wide margin. But recover the card ($8,565 launch MSRP, March 2025; street lists have since drifted higher) against that saving and the break-even sits at billions of output tokens per month. Most agents never spend anywhere near that on tokens.

The reasons a self-hosted 96GB lane is still the right call for serious agent work are the ones the meter does not charge for:

261,494-token context, measured. The full native window, with a correct planted-needle retrieval at TTFT 99.95 s. No prompt-size tier, no context fee curve.
No meter on concurrency. 408.62 aggregate tok/s at c4 with no per-request rate limit and no queue behind another provider's scheduler.
The prompt never leaves the building. For operator data, client code and trading logic, that clause is worth more than the kWh.
One thermal gate to respect. 91°C at saturation is the ceiling: hold load, power, or cooling fixed at 90°C and the lane is stable. That is a maintenance rule, not a bill.

The practical rule: run the lane when the workload is context-heavy, concurrency-heavy or private, and keep a metered API for the burst you would rather not self-host. Electricity decides the tie, not the match.

What is not verified here

Wall power: only the GPU board was metered; system overhead is an estimate.
Latency distribution: medians are reported; p95 and p99 were not collected this run.
API comparison uses published list prices; negotiated and batch rates differ.
One model, one run, one workload mix. Other checkpoints and mixes shift the split.

The full method, the two-lane configs, and the thermal log are inthe Blackwell field guide. Every other number the Lab publishes is filed underLocal LLM Inference.