/learn
From one GPU to a hall of racks
One chapter per rung of the machine, each opening with an animation. Every frame is computed by this site's model (checked against its Python reference), and every hardware figure links its source. Toggle layers (Concept / Maths / Code) inside any chapter to choose how deep to go. What happens inside one GPU is on GPU Kernels Explained; for slides on the same hardware, see the NVIDIA GPU series.
The ladder from one GPU package to a hall of racks: how many GPUs each rung holds, and how fast one GPU can move data at each.
Compute dies and HBM stacks on one package: why memory bandwidth and capacity set the shape, and how many GPUs a model's weights need.
A compute tray's GPUs, Grace CPUs and network cards, and the paths data takes between them: network, CPU memory, another GPU.
72 GPUs, 18 switch chips, one link from every GPU to every chip: why an NVL72 rack behaves like one big GPU for all-reduce and all-to-all.
Leaf, spine and core switches, rail-aligned fabrics, oversubscription, and the transceivers every link needs.
- 06Power
Power shelves, busbars and the move to 800 V DC; why a training job's synchronised swings need storage in the rack; tokens per megawatt.
- 07Cooling
Why air runs out at rack scale: cold plates, the coolant loop, CDUs and facility water, sized with Q = ṁ c_p ΔT.
Prefill and decode as separate pools in one NVL72-sized rack, the KV hand-off between them, pool sizing and context-optimised parts, run live on a tested simulator.
How a training job's tensor, pipeline and data parallelism land on NVLink domains and the network, and how often a large job is interrupted.
TPU pods built from torus cubes and optical switches, and other vendors' rack-scale systems: what each changes about the ladder.
Capital, electricity and utilisation turned into cost per million tokens and tokens per megawatt, with every price marked illustrative.