15 chapters in five parts. Every hardware figure is sourced to the vendor, standards body or published result it came from, and each chapter ends with its sources.
Foundations
Hardware
Accelerators: GPUs, TPUs and Beyond
GPU architecture for practitioners, NVIDIA, AMD, Intel and Google TPU generations with datasheet specs, the FP64 question, partitioning, form factors and a buying guide for research centers.
ReadMemory and Storage
How memory bandwidth and capacity now bound most AI and HPC work, what HBM, unified memory and CXL actually give you, and how to size parallel file systems, local NVMe, checkpoints and data pipelines for a research center.
ReadInterconnects: Scale-Up and Scale-Out
NVLink, UALink, PCIe, InfiniBand, Ethernet for AI and Slingshot explained for practitioners, with collectives, congestion control, testing commands and design guidance for 8, 64 and 512+ GPU clusters.
ReadPower, Cooling and Facilities
What changed when GPU racks went from 10 kW to more than 130 kW, and how to plan power, cooling, floor, water and procurement for your first liquid-cooled systems.
ReadSoftware and Operations
Systems Software and Provisioning
How a 2026 AI + HPC cluster gets from powered-off hardware to a usable, secure software environment: OS choices after CentOS, Warewulf and friends, Redfish, the NVIDIA and AMD GPU stacks and their compatibility rules, Spack, EasyBuild, Lmod, Python tooling, containers and identity.
ReadScheduling and Orchestration
How to schedule GPUs well: Slurm GRES, binding, topology, fairshare, QOS and preemption, working sbatch recipes for PyTorch and MPI, Kubernetes with Kueue and the GPU Operator, GPU sharing, Open OnDemand and workflow managers.
ReadObservability, Performance and Efficiency
GPU telemetry, job accounting, health checks, fabric monitoring, profilers, the metrics that matter (MFU, SM activity, TTFT), energy accounting, benchmarks and reliability at scale, with working commands.
ReadResearch Computing Operations
Service and cost models, national resources (ACCESS, NAIRR, INCITE and ALCC), facilitation and training, GPU onboarding, CUI, HIPAA, export control and CMMC, capacity planning for AI demand, staffing, sustainability and measuring impact.
ReadWorkloads
Training at Scale
How large models are actually trained on GPU clusters: where the memory and time go, how DP, FSDP, TP, PP, CP and EP combine and map onto hardware, which frameworks to use, and how to survive failures at scale. Includes a tested FSDP2 skeleton and a multi-node Slurm launch script.
ReadInference at Scale
How LLM inference actually works on GPUs, why decode is bound by memory bandwidth, and how to size, serve, benchmark and budget models on a research cluster with vLLM, SGLang, TensorRT-LLM and friends.
ReadScientific Computing in the AI Era
Where classic simulation stands in 2026, how to program and port codes to GPUs, GPU-aware MPI, the major GPU applications, and how AI surrogates, weather models and protein structure predictors fit alongside them.
ReadCloud and Hybrid AI/HPC
Cluster tooling on AWS, Google Cloud and Azure, the GPU and TPU instance families, capacity models from spot to reserved blocks, neoclouds, cost guardrails for research, hybrid bursting from campus Slurm, security, and how to reason about cloud versus on-prem.
ReadReference
Linux Essentials
Files, permissions, processes, Vim, the shell, Screen, SSH and archives: the 2019 reference chapter, still true.
ReadGlossary
Plain definitions of more than 100 terms spanning accelerators, memory, interconnects, scheduling, training, inference and numerics, with links to the chapters that cover each one in depth.
ReadThe 2019 edition
The original HPC Field Manual, written in 2018-2019 for CPU clusters, is preserved as written. The Linux chapter carries forward into this edition.