Power

Power shelves, busbars and the move to 800 V DC; why a training job's synchronised swings need storage in the rack; tokens per megawatt.

Loading the animation…

Concept

A GB200 NVL72 rack draws about 120 kW, NVIDIA's own figure in the DGX GB200 user guide. Spread over its 72 GPUs that is 1.67 kW each, against a Blackwell GPU's maximum of 1.2 kW: the rest goes to the Grace CPUs, the NVLink switch trays, network cards, fans and conversion losses.

Power shelves. The rack does not take mains power into each tray. 8 power shelves, each holding 6 supplies of 5.5 kW, convert AC to a DC bus of about 50 V. That is 264 kW of supplies, arranged N+N: either half can carry the rack, so 132 kW survives the loss of half of them, 1.1 times the rack's draw.

The busbar. At 50 V, 120 kW is 2,400 A, carried by copper bars down the back of the rack. Current is what costs copper and heat: the loss in a conductor is I²R, so for the same power it falls with the square of the voltage. NVIDIA's plan for racks of 1 MW (stated for 2027, so planned, not shipping) is to bring 800 V DC to the rack instead of 54 V: it says a 1 MW rack at 54 V would need up to 200 kg of busbar copper.

Rack power	Bus	Current	Loss vs 54 V	Copper (derived)
120 kW	50 V	2,400 A	1.17×	25.9 kg
120 kW	54 V	2,222 A	1×	24 kg
120 kW	800 V	150 A	0.00456×	1.62 kg
1 MW	54 V	18,519 A	1×	200 kg
1 MW	800 V	1,250 A	0.00456×	13.5 kg

The copper column is our extrapolation, not NVIDIA's: copper in proportion to current, at the current density of NVIDIA's 1 MW, 54 V example.

Power smoothing. Training is synchronous. Every GPU computes, then every GPU waits for the same all-reduce, so a whole hall's power rises and falls together; the Llama 3 paper reports fluctuations "on the order of tens of megawatts". The animation shows one rack's share. With no storage, the grid sees the full swing. GB300 NVL72's power supplies hold 65 J per GPU in capacitors (NVIDIA's figure). If the load swings every 0.05 s, that store flattens the grid's draw completely and cuts the peak by 25%. If it swings every quarter of a second, the store runs empty in each high phase and the cut falls to 16%. At one second it is 4%. Capacitors are for sub-second ripple; slower swings need the other tools NVIDIA describes (power caps that ramp gradually, and a "power burner" that keeps GPUs drawing while the job pauses) or batteries outside the rack. NVIDIA reports a 30% cut in peak grid demand on a real training run; that is its measurement, quoted, not this model's output.

Tokens per megawatt. Power is now the scarce input for many AI data centres, so serving efficiency is often quoted per megawatt. Chapter 8's simulated rack produces 850,011 output tokens a second per megawatt of GPU power, using the simulator's illustrative power coefficients; chapter 11 puts the whole rack's power and a facility overhead on top. Vendors quote this metric too: NVIDIA says Vera Rubin NVL72 "delivers up to 10x more tokens per megawatt than NVIDIA GB200 NVL72", a claim we attribute and do not model.

Maths

A rack alternates between $P_{\text{hi}}$ for $t_{\text{hi}}$ and $P_{\text{lo}}$ for $t_{\text{lo}}$ , with stored energy $E$ . Holding the grid flat at $P_g$ through the high phase takes $(P_{\text{hi}} - P_g)\,t_{\text{hi}} \le E$ from the store, so the lowest flat level is

P_g = \max\!\left(\bar P,\; P_{\text{hi}} - \frac{E}{t_{\text{hi}}}\right), \qquad P_{\text{low phase}} = P_{\text{lo}} + \frac{(P_{\text{hi}} - P_g)\,t_{\text{hi}}}{t_{\text{lo}}},

where the second is the draw that refills the store in the low phase. When $E \ge (P_{\text{hi}} - \bar P)\,t_{\text{hi}}$ both levels equal the average $\bar P$ : the grid sees no swing at all.

On the bus, current and resistive loss for power $P$ at voltage $V$ are

I = \frac{P}{V}, \qquad P_{\text{loss}} = I^2 R \;\Rightarrow\; \frac{P_{\text{loss}}(V)}{P_{\text{loss}}(54\ \text{V})} = \left(\frac{54}{V}\right)^{2}.

Code

The smoothing model, cut from src/lib/racks/model.ts:

const period = tHi + tLo;
const pAvg = (pHi * tHi + pLo * tLo) / period;
const level = Math.max(pAvg, pHi - eStore / tHi);
const used = (pHi - level) * tHi;
const low = pLo + used / tLo;