About this site

AI Racks Explained shows the physical machines that train and serve large models, from one GPU package to a hall of racks, one rung per chapter, each around an animation. It sits in the companion sites' Hardware track beside GPU Kernels Explained and Systolic Arrays Explained, and builds on the Training and Serving sites. Chapters link the matching slides of the NVIDIA GPU series. NVIDIA's rack-scale systems are the main worked example because their specifications and reference designs are published in detail; chapter 10 covers other designs. Chapter 8, on disaggregated inference, runs a separate, tested simulator in the browser.

The model

reference/rack_model.py is a teaching model, not a simulator of any product. It computes the scale ladder (GPUs and bandwidth per GPU at each rung), what an HBM package holds and how fast it can be read, chunked store-and-forward transfers along a chain of links, the wiring and bandwidth of an NVLink domain, ring all-reduce on real numbers, and the switches, links and transceivers of fat-tree fabrics, including the rail-aligned rules of a published reference architecture. For chapters 6 to 11 it adds a rack's power shelves and bus currents, power smoothing with stored energy, the coolant loop from Q = ṁ c_p ΔT, where a training job's traffic lands on NVLink domains, checkpoint intervals by Young's rule, hop counts and bisections of a torus, and cost per token.

Chapter 8 does not use this model. It runs Disaggregated_Inference_Sim's own JavaScript engine, vendored byte for byte at commit ba37774, in a Web Worker, on the simulator's GB200 preset (one GPU of a GB200 superchip, from NVIDIA's per-GPU figures; its section 28 compares it with the H100, H200 and B200 presets). scripts/disagg_reference.py runs the simulator's Python package at the same commit on the same requests, and the unit tests require the browser engine to reproduce every request's timestamps bit for bit and every recorded summary the chapter quotes. The simulator's own validation against published serving results is section 22 of its results.md.

• Every time is a bandwidth term: latency, protocol overhead, congestion and achieved-versus-peak efficiency are not modelled.
• NVIDIA quotes NVLink and NVLink-C2C bandwidth as the sum of both directions; the model works per direction (half) and says which it prints.
• Weights-only memory sizing ignores the KV cache, activations and the runtime; FP4 is counted at half a byte per weight, without its block scales.
• Optics power uses a watts-per-transceiver you choose: it is illustrative, because the sources used do not publish per-port power.
• Power smoothing assumes a square-wave load and a perfect store; the coolant loop takes an illustrative share of heat on cold plates and illustrative temperature rises; checkpoint intervals apply one published run's interruption rate to every job size; every price in the cost chapter is illustrative.

How hardware figures are sourced

Every figure is a field with a value, a status and a source, read on 2026-10-13 (the full list). Vendor product pages, documentation, reference architectures and technical blogs come first; papers are checked by DOI or arXiv ID. Where the sources used do not give a figure, the site says “not disclosed” rather than estimating it. Products whose specifications the vendor marks preliminary, or gives a future date for, are labelled. Marketing claims (“up to N×”) are quoted and attributed, never used as model inputs.

Spec cross-checks, rules and a measurement

The model is compared with published figures that it does not take as inputs, and each comparison is labelled by what it actually tests. Most are spec cross-checks (13 of them): the model's arithmetic on a vendor's per-GPU or per-chip figures against the same vendor's rack or system total (rack NVLink and HBM totals, a switch tray's bandwidth, AMD's and AWS's system totals). They show that the specifications agree with each other and that the model reads them correctly; they do not show that it predicts performance. Rules (14) test a structural rule the model applies: the NVL72 wiring (links equal switch ports), the size of a two-tier fat tree, and the leaf, spine and core switch counts of the published SuperPOD design, which the model derives from the rail-aligned rules rather than reading. Across these the model is between 0.9874 and 1.0368 of the published value. The largest gaps are rounding in the published figures: Vera Rubin NVL72's rack HBM bandwidth (1,400 TB/s) is 1.3% more than 72 times its per-GPU figure, and “over 10,000 GPUs” for a two-tier fabric of 144-port switches is the model's 10,368 rounded down.

One row is a measurement: NCCL's in-switch (NVLS) all-reduce of large messages on an eight-GPU NVLink 4 node, 276 GB/s in a 2026 study, against the model's in-switch bound of 448.9 GB/s: the bound is 1.63 times what the software reaches. Chapter 4 also takes the latency term of its collectives from the same measurements. The rack simulator of chapter 8 has its own validation against published serving results (above).

How it is checked

• Reference tests. tests/python/test_rack_model.py checks every field has a known source and status, the ladder, the package and fit sums, the store-and-forward pipeline against its closed form, ring all-reduce ending with every GPU holding the sum, the fat-tree sizes against Al-Fares et al. (k³/4 hosts with 5k²/4 switches), and the SuperPOD counts.
• Exact parity. The TypeScript port, src/lib/racks/model.ts, repeats every function with the same operations in the same order; scripts/make_fixtures.py writes the reference's results over every parameter the widgets offer, and the unit tests require the port to reproduce all of them exactly, with no tolerance. CI fails if the fixtures, the site data or results.md are out of date.
• Animations from the model. Every animation draws a sequence of states the model computes; a frame is a pure function of one state. The end-to-end tests set chosen frames of every animation and require the caption to match the caption built from the Python reference's state for that frame.
• Numbers in the prose are printed from the model when the page is built, not typed.

The animations

Every animation has play and pause, step back and forward, a scrub bar, speeds from 0.25× to 4× and reset; with the animation focused, Space plays or pauses and the arrow keys step. Each step has a one-line caption, also announced to screen readers. With reduce motion set in your system, nothing plays by itself. Animations pause when scrolled out of view. Colours come from Okabe and Ito's colour-blind-safe palette, the same in light and dark mode: HBM purple (as on every companion site), scale-up links (NVLink, NVLink-C2C) green, the scale-out network sky blue, PCIe orange, CPU memory yellow; waiting is always a hatched pattern as well as a warning colour.

Source

The code, the model and the tests are on GitHub (MIT licence). The design system is copied from the companion sites; the README records where each piece came from.