The Convergence

When I wrote the first HPC Field Manual in 2018, a research cluster was a rack of dual-socket CPU nodes, an InfiniBand fabric, a Lustre or GPFS scratch file system, and Slurm. GPUs were a partition in the corner that a few molecular dynamics and deep learning groups fought over. In 2026 that picture has flipped. The largest systems on the planet are GPU or APU machines, the same hardware trains frontier language models and runs climate codes, and the question a facilitator hears most often is not "how many cores?" but "how much GPU memory, at what precision?" This chapter explains how that happened, what the two scoreboards measure, why numeric precision is now a first-order design decision for scientists, and which parts of the craft have not changed at all. It closes with a roadmap of the rest of the manual.

From MPI Clusters to GPU-Dense Systems

The CPU cluster era

For roughly two decades, the standard recipe for research computing was commodity x86 servers glued together with MPI. Performance came from more nodes and more cores, and the programming model was message passing between ranks with some OpenMP threading inside each node. The TOP500 list, which started in 1993 and has been published every June and November since, tracked that growth.[1] That model still describes a large share of campus workloads, and nothing in this manual argues it is obsolete. What changed is where the floating-point operations live.

Accelerators take over the top of the list

By the time Frontier arrived, GPUs were already established at the top. The May 2022 TOP500 announcement, which recorded Frontier as the first system to exceed one exaflop/s on HPL (1.102 Exaflop/s), listed Summit at No. 4 with six NVIDIA V100 GPUs per node and Sierra at No. 5 with four V100s per node.[3] Frontier itself paired one 64-core AMD EPYC CPU with four AMD Instinct MI250X GPUs per node across 9,856 nodes, for 53 TF of double precision per node.[4] The CPU had become the host; the accelerators did the math.

Three forces drove this:

• Power. Dense matrix units deliver far more floating-point work per watt than general-purpose cores. Every exascale design is, first, a power budget.
• Memory bandwidth. GPUs sit on stacks of high bandwidth memory (HBM). A 2020-era A100 already offered 1,555 GB/s from 40 GB of HBM2.[11] Many simulation kernels are bandwidth-bound, so this matters as much as FLOPS (see the roofline section below).
• Money following AI. Once deep learning became the largest market for accelerators, vendors put their silicon, software and networking investment there. Scientific computing now rides on hardware whose feature list is set largely by AI.

The exascale generation in 2026

The June 2026 TOP500 (the 67th edition) lists five systems above one exaflop/s on HPL.[1] Four of them are accelerator machines built for both simulation and AI; the new No. 1 is a CPU-only design, which is a useful reminder that there is more than one path.

Rmax values are from the June 2026 TOP500 press release.[1] A few details worth knowing about each:

• El Capitan uses the AMD MI300A, an APU that puts CPU cores and GPU compute dies in one package with shared HBM. LLNL lists 61.3 TFLOPS of vector FP64 and 122.6 TFLOPS of matrix FP64 per MI300A.[5] It reports 60.94 GFLOPS/W on HPL.[1]
• Aurora has 10,624 nodes, 63,744 GPUs, 20.4 PB of aggregate memory, and a 230 PB DAOS storage system rated at 31 TB/s.[6]
• JUPITER Booster is Europe's first exascale system and runs on Eviden BullSequana XH3000 direct liquid-cooled hardware.[2]
• LineShine reached 2.198 EF/s at about 42.2 MW, or 52.07 GFLOPS/W, with 13.79 million cores.[1]

AI supercomputers and the cloud

Alongside national lab machines, purpose-built AI systems and cloud clusters now sit on the same list. Microsoft's Azure Eagle system (Intel Xeon with NVIDIA H100) is No. 7 in June 2026 at 561.2 PF/s.[1] The next generation of lab systems is being designed explicitly for both roles. NVIDIA's June 2026 announcement of Vera Rubin for scientific computing names NERSC's Doudna, LRZ's Blue Lion (scheduled for 2027) and Los Alamos's Mission, Vision and Veritas systems, and describes Veritas as "designed for agents".[7] These are announced systems, not yet on the list.

The line between "AI supercomputer" and "HPC supercomputer" is now mostly a question of who paid for it and what the procurement document emphasized. The hardware underneath - accelerators with HBM, a scale-up fabric inside the node or rack, a scale-out fabric between them, a parallel file system, Linux and a batch scheduler - is the same.

Field note: When faculty ask whether they need "an AI cluster" or "an HPC cluster," the honest answer in 2026 is usually "one cluster with a few different partitions." Size the GPU memory, interconnect and storage for the most demanding workload you plan to support, and let the scheduler carve it up. Buying two separate islands doubles the operations cost and halves the utilization.

The Two Scoreboards: TOP500 and MLPerf

TOP500: HPL, HPCG and HPL-MxP

The TOP500 ranks systems by the High Performance Linpack (HPL) benchmark, which solves a dense linear system in 64-bit floating point. HPL rewards peak FP64 matrix throughput and a well-tuned interconnect. It has well-known limits: real applications rarely look like a dense LU factorization. Two companion benchmarks fill in the picture:

• HPCG exercises sparse, memory-bound access patterns closer to real solvers. In June 2026 LineShine leads at 22.00 PF/s, ahead of El Capitan at 17.41 and Fugaku at 16.00.[1] Note how small those numbers are relative to HPL: that gap is the memory wall in one line.
• HPL-MxP (formerly HPL-AI) solves the same problem but lets the LU factorization run in low precision, then uses iterative refinement to recover a 64-bit accurate answer.[20] El Capitan leads at 16.7 EF/s, a 9.2x speedup over its HPL score; Aurora (11.6 EF/s, 11.5x) and Frontier (11.4 EF/s, 8.4x) follow. LineShine, being CPU-only, posts 7.92 EF/s, a 3.6x speedup.[1]

HPL-MxP is the benchmark that most directly measures the convergence: it asks how much of a machine's AI-oriented low-precision hardware can be turned into a scientifically valid FP64 answer.

MLPerf: training and inference

MLPerf, run by the MLCommons consortium, measures time-to-train and inference throughput on fixed reference models with accuracy targets. The Closed division holds the model fixed so results are comparable; the Open division allows other optimizations.[10] Recent rounds show where AI workloads are going:

• Training v6.0 (June 2026) added DeepSeek V3 (671 billion total parameters, 37 billion active per token) and GPT-OSS 20B, both Mixture-of-Experts models. Submitters used 95 unique systems with thirteen different accelerators; 60 percent of systems were multi-node, and cloud submissions more than doubled compared with v5.1. Submitters used multiple different FP4 training recipes.[8]
• Inference v6.0 (April 2026) added GPT-OSS 120B, DLRMv3, a text-to-video test and a vision-language benchmark. The largest system had 72 nodes and 288 accelerators.[9]
• Inference v6.1 (September 2026) drew a record 30 submitters and 120 systems, added end-to-end RAG and agentic edge benchmarks, and set a scale record with a 512-GPU submission. MLCommons attributes part of the gains to some submissions moving from FP8 to FP4 while meeting accuracy requirements. The largest per-accelerator jumps came from NVIDIA Vera Rubin systems submitted in the Preview category.[10]

Reading the scoreboards

TOP500 tells you about FP64 capability and fabric quality. MLPerf tells you about low-precision throughput, software maturity and scale-out for AI. Neither tells you how your users' codes will run. Use both as sanity checks against vendor claims, then benchmark your own workloads (see Observability, Performance and Efficiency).

Field note: When a vendor quotes a TFLOPS number, ask three questions before you write it in a proposal: which precision, dense or sparse, and is it a single GPU or a whole rack? The same product page can show numbers that differ by 100x depending on those answers.

A Workload Taxonomy for 2026

A modern research computing center supports at least seven kinds of work on the same hardware. They stress the system differently, and they need different scheduling, storage and support.

The precision and bottleneck columns are general patterns, not rules; individual codes vary.

Simulation

Classic tightly coupled codes: domain decomposition, halo exchanges, implicit solvers, FFTs. These are the workloads HPL and HPCG stand in for. Most still require FP64 for stability or for results that match validated references. Many have been ported to GPUs over the past decade, often with help from the DOE Exascale Computing Project, which prepared applications for Aurora and its peers.[6]

Training and fine-tuning

Pre-training a large model is the most demanding synchronous workload most centers will ever see: thousands of accelerators, all-reduce and all-to-all collectives every step, and checkpoint writes measured in terabytes. Few campuses pre-train frontier models, but many support fine-tuning of open-weight models, where the main constraint is fitting weights, gradients and optimizer state into GPU memory.

Inference

Serving a model is dominated by memory bandwidth during token generation and by KV cache capacity for long contexts. MLPerf's move to multi-node and 512-GPU inference submissions shows that inference is now a scale-out workload in its own right.[10]

Data analytics

Dataset preparation, tokenization, deduplication, image preprocessing and post-simulation analysis. Often CPU-heavy and I/O-heavy. These jobs are where a parallel file system is either adequate or a disaster.

AI-for-science surrogates

Machine-learned models trained on simulation output or observational data, then used in place of (or alongside) the expensive solver. GraphCast is the reference example: trained on reanalysis data, it predicts hundreds of weather variables over 10 days at 0.25 degree resolution in under one minute, and outperformed the most accurate operational deterministic systems on 90 percent of 1,380 verification targets.[28] Surrogates blur the line between simulation and training: the same group runs FP64 solvers to generate training data and BF16 training to build the model.

Agentic and LLM workflows in research

The newest category: researchers running LLM agents that call tools, launch jobs, search literature, write code and analyze results. MLPerf Inference v6.1 introduced agentic and end-to-end RAG benchmarks for exactly this reason.[10] The US DOE Genesis Mission, launched by executive order in November 2025, aims to connect national lab supercomputers, AI systems and scientific instruments into an integrated discovery platform.[27] For a campus center, agentic workloads look like a swarm of small, latency-sensitive inference calls plus bursts of ordinary batch jobs, which is an awkward fit for a queue designed around multi-day MPI runs.

The Numeric Precision Spectrum

If there is one concept a research computing professional must understand in 2026, it is floating-point precision. It decides which hardware a workload can use, how fast it runs, and whether the answer is trustworthy.

How the formats differ

A floating-point number has a sign bit, exponent bits (which set the dynamic range) and mantissa bits (which set the precision). The formats below are listed from widest to narrowest.

Sources for the table: FP64 significand from NVIDIA's cuBLAS emulation documentation[18]; FP32 and BF16 mantissa bits from the PyTorch documentation[17]; TF32 from the NVIDIA Ampere architecture description[11]; BF16 exponent from Google[12]; FP16 mantissa from NVIDIA[11] and its 5-bit exponent from the FP8 paper, which describes E5M2 as IEEE half precision with fewer mantissa bits[13]; FP8 encodings from the FP8 paper[13]; FP4 E2M1 from NVIDIA[15]. The 11 exponent bits and 52 stored bits of FP64 match the IEEE 754 binary64 layout, and you can check them yourself:

import numpy as np

for t in (np.float64, np.float32, np.float16):
    f = np.finfo(t)
    print(t.__name__, "bits", f.bits, "exp", f.nexp, "mantissa", f.nmant, "eps", f.eps)

# float64 bits 64 exp 11 mantissa 52 eps 2.220446049250313e-16
# float32 bits 32 exp 8 mantissa 23 eps 1.1920929e-07
# float16 bits 16 exp 5 mantissa 10 eps 0.000977

The 16-bit and TF32 formats

BF16 was designed at Google Brain for TPUs. It keeps FP32's 8-bit exponent, so it has the same dynamic range as FP32, and gives up mantissa bits. On TPUs, each multiply uses bfloat16 and the accumulation is done in FP32.[12] That combination (low-precision multiply, higher-precision accumulate) is the pattern behind nearly all AI hardware.

TF32 arrived with the A100. Tensor cores read FP32 inputs, compute internally with the FP32 range and FP16-level precision (10 mantissa bits), and write FP32 outputs.[11] In PyTorch it is opt-in for matrix multiplications through torch.set_float32_matmul_precision: "highest" (the default) keeps full FP32, while "high" allows TF32 or a two-BF16 decomposition.[17]

import torch

# Default: full FP32 internal precision for float32 matmuls
torch.set_float32_matmul_precision("highest")

# Opt in to TF32 (or bf16x2 where available) for float32 matmuls
torch.set_float32_matmul_precision("high")

This matters for scientists who use PyTorch or JAX as a numerical library rather than for deep learning: know which setting your code runs with before you publish numbers from it.

FP8, FP4 and microscaling

The FP8 proposal from NVIDIA, Arm and Intel defines two encodings: E4M3 (more precision) and E5M2 (more range). The authors showed FP8 training matching 16-bit results on models up to 175 billion parameters without changing hyperparameters.[13]

Below 8 bits, a single per-tensor scale factor is not enough. Microscaling (MX) formats attach a shared scale to small blocks of values. The OCP MX specification v1.0, backed by AMD, Arm, Intel, Meta, Microsoft, NVIDIA and Qualcomm, defines MXFP8, MXFP6, MXFP4 and MXINT8.[14] MXFP4 uses one power-of-two scale per 32-value block. NVIDIA's NVFP4 uses a smaller 16-value block with an FP8 (E4M3) scale plus a per-tensor FP32 scale, for about 4.5 bits per value. NVIDIA reports roughly 3.5x lower memory footprint than FP16, and 1 percent or less accuracy degradation when DeepSeek-R1-0528 was quantized from FP8 to NVFP4.[15] AMD's MI355X also lists MXFP6 and MXFP4 support.[23]

Why precision matters differently for science and AI

Neural network training is statistically noisy by design; stochastic gradient descent tolerates rounding error the way it tolerates minibatch noise. Many scientific computations are not like that. Long time integrations accumulate error, ill-conditioned linear systems amplify it, and conservation laws must hold to tight tolerances. A climate model that drifts, or an eigenvalue solver that converges to the wrong answer, is worse than a slow one.

That is why the precision trend in accelerator design is a real concern for science. For years FP64 grew with each generation: a single H100 SXM lists 34 TFLOPS of FP64 and 67 TFLOPS of FP64 Tensor Core.[21] Then look at NVIDIA's HGX specification table. An 8-GPU HGX B200 lists 296 TFLOPS of FP64/FP64 Tensor Core. The Blackwell Ultra HGX B300, with the same TF32 tensor rating (18 PFLOPS, which NVIDIA footnotes as a sparse figure), lists 10 TFLOPS of FP64 for all eight GPUs.[22] The HPL-MxP authors describe this directly as a performance inversion: the move from Hopper to Blackwell lowers performance in the IEEE 64-bit format.[20] For a center buying hardware, the specific SKU now matters as much as the generation.

Field note: Before you buy AI-oriented GPUs for a mixed community, inventory which of your heavy users need native FP64. Ask them to run their code with a representative input on a loaner or cloud instance of the exact SKU. The spec sheet FP64 line is the single most common surprise in 2025-2026 procurements.

Mixed Precision and Emulated FP64

Mixed-precision training

The 2017 mixed-precision training paper set out the recipe AI still uses: keep an FP32 master copy of the weights, use loss scaling to keep small gradients representable, and do half-precision arithmetic that accumulates into FP32.[16] BF16 largely removed the need for loss scaling because it has FP32's range. FP8 and FP4 training bring scaling back, now per tensor or per block.[13][15] Details and framework settings are in Training at Scale.

Mixed-precision solvers and iterative refinement

Numerical analysts have their own version: factor the matrix in low precision, then iteratively refine the solution using residuals computed in high precision. That is exactly what HPL-MxP does, recovering 64-bit accuracy from a low-precision LU.[20] Where a code's inner loop is a dense solve or can be preconditioned by one, this pattern can turn AI-oriented matrix units into science throughput, and the HPL-MxP speedups on El Capitan, Aurora and Frontier show the scale of the opportunity.[1]

Emulating FP64 on low-precision units

A second approach leaves the algorithm alone and emulates FP64 matrix multiplication itself. The Ozaki scheme, proposed in 2012, splits FP64 operands into slices that low-precision units can multiply exactly and then sums the partial products; later work extended it to integer and FP8 tensor cores.[19]

This is no longer a research curiosity. cuBLAS in CUDA Toolkit 13.0 Update 2 added FP64 matmul emulation on Blackwell INT8 tensor cores, alongside BF16x9 emulation of FP32 on BF16 tensor cores. An automatic dynamic precision framework inspects the inputs, picks emulation parameters to give accuracy equal to or better than native FP64, and falls back to native execution when emulation would not help. NVIDIA reports a 1.5x end-to-end speedup for Quantum ESPRESSO with the default automatic settings on an RTX PRO 6000 Blackwell Server Edition, and nearly 3x with a manually reduced 39 mantissa bits, with output values consistent to 12 significant digits.[18]

Vendors are now building this into their specifications. NVIDIA's preliminary HGX Rubin NVL8 table lists 265 TFLOPS of native FP64 and 1,600 TFLOPS of FP64 DGEMM for eight GPUs, footnoting the DGEMM figure as peak performance using tensor-core emulation algorithms; the per-GPU figures are 33 and 200 TFLOPS.[22] Rubin is announced, with specifications marked preliminary; it appeared in MLPerf Inference v6.1 only in the Preview category.[10]

Two cautions. Emulation accelerates matrix multiplication (GEMM), not the vector, stencil and sparse operations that dominate many codes, so a code's speedup depends on how much of its time is in dense matmul. And because the Ozaki scheme needs a data-dependent number of mantissa bits, performance varies with the input.[18] Profile before you promise anyone a speedup. Scientific Computing in the AI Era covers porting strategy in depth.

Field note: Treat emulated FP64 like any other numerical change: rerun your validation suite, compare against a native-FP64 reference to the digits your field cares about, and record the library version and emulation settings in the paper's methods section. Reviewers will ask.

The Roofline Model and the Memory Wall

The roofline model

Williams, Waterman and Patterson's roofline model relates a processor's attainable performance to its off-chip memory traffic.[24] The key quantity is arithmetic intensity: floating-point operations performed per byte moved from memory.[25] Attainable performance is the lesser of peak compute and (arithmetic intensity x memory bandwidth). The crossover point, peak FLOPS divided by bandwidth, is the ridge point. Kernels left of the ridge are memory-bound; kernels to the right are compute-bound.

A worked example

Take the AMD Instinct MI355X, whose product page lists 78.6 TFLOPS peak FP64, 2.5 PFLOPS peak dense BF16 matrix, and 8 TB/s of HBM3E bandwidth.[23] Dividing peak by bandwidth gives:

• FP64 ridge point: 78.6 TFLOPS / 8 TB/s = about 9.8 FLOP/byte.
• BF16 dense ridge point: 2,500 TFLOPS / 8 TB/s = about 312 FLOP/byte.

Now compare two kernels:

• A STREAM-style triad in FP64 (a[i] = b[i] + s * c[i]) does 2 FLOPs per element while moving 24 bytes (two 8-byte reads, one 8-byte write): about 0.083 FLOP/byte. At 8 TB/s that caps out near 0.67 TFLOPS, under 1 percent of the FP64 peak. No amount of extra FLOPS helps this code; only bandwidth does.
• LLM token generation at batch size 1 with BF16 weights reads each 2-byte weight once for one multiply-add (2 FLOPs): about 1 FLOP/byte, two orders of magnitude left of the BF16 ridge point. This is why decoding is memory-bound and why serving systems batch requests aggressively, as Gholami and colleagues analyze for decoder models.[25]

These are idealized bounds from vendor peak numbers, not measurements; real kernels land below the roof. The value is in the shape: for both a sparse CFD solver and a chatbot, the limiting resource is memory bandwidth.

The memory wall

The term "memory wall" dates to Wulf and McKee in 1995. Gholami and colleagues quantify the modern version: over the past 20 years, peak server FLOPS grew about 3.0x every two years, while DRAM bandwidth grew 1.6x and interconnect bandwidth 1.4x over the same interval.[25] Every generation, the ridge point moves further right and more kernels become memory-bound.

Hardware responses include wider and faster HBM (the JEDEC HBM4 standard, published April 2025, defines transfer speeds up to 8 Gb/s across a 2048-bit interface, for total bandwidth up to 2 TB/s[26]), larger on-die caches, and fast scale-up fabrics such as NVLink, whose sixth generation lists 3.6 TB/s per GPU on the announced Rubin platform (a preliminary specification).[31] Software responses include kernel fusion, IO-aware algorithms, lower-precision storage (which cuts bytes moved as well as FLOPs), and batching. The memory hierarchy has its own chapter: Memory and Storage. Interconnects are in Interconnects.

Field note: Teach every new GPU user to compute the arithmetic intensity of their hottest kernel on a whiteboard before they ask for more GPUs. Half the time the answer is "you are bandwidth-bound; more GPUs of the same kind will scale your cost faster than your speed."

How the Research Computing Center Has Changed

A new user base

Ten years ago, a typical campus HPC user came from physics, chemistry, engineering or genomics, and knew what a job script was. In 2026, a large and growing share of new accounts come from computer science, linguistics, economics, social sciences, medicine and the arts, and their first question is how to run a notebook or a model on a GPU. Many have never used a terminal. They arrive with Python environments, container images and model checkpoints, not Fortran and Makefiles.

A new mission

The center's job has widened from "run the cluster" to something closer to "run the campus AI and computing platform." In practice that means:

• Serving, not just batch. Hosting shared inference endpoints and interactive sessions alongside the queue, with very different latency expectations (see Inference at Scale and Scheduling and Orchestration).
• Data governance. Model weights, licensed datasets and sensitive data (human subjects, controlled unclassified information) all need policy and technical controls (see Research Computing Operations).
• Hybrid capacity. Few campuses can buy enough GPUs for peak demand; cloud and national resources fill the gaps (see Cloud and Hybrid AI/HPC).
• Facilities. GPU racks drive power density and liquid cooling requirements that most campus data centers were not built for. JUPITER, the Green500 leaders and the announced Vera Rubin systems all use direct liquid cooling.[1][2][7] See Power, Cooling and Facilities.
• Facilitation at a different level. Teaching precision, memory budgets and parallelism strategy, not just sbatch.

Utilization and fairness

GPU capacity is the most expensive resource most centers operate, so idle GPUs are the most expensive waste in the building, and fair-share policies designed around core-hours need rethinking for GPU-hours, GPU memory and partitioned GPUs. Observability covers how to measure it; Scheduling covers how to enforce it.

What Stays the Same

For all the change, the foundations of the 2019 manual still hold. That is good news for anyone with a decade of cluster experience.

• Linux. Leadership systems still run Linux distributions. The June 2026 top ten includes systems running Red Hat Enterprise Linux (JUPITER Booster and Eni's HPC7) and Kylin OS (LineShine).[2][32] The Linux Essentials chapter from 2019 is still accurate.
• Batch schedulers. Slurm is still doing the job it did in 2019: an open-source cluster manager and job scheduler for Linux clusters that allocates resources, launches and monitors parallel jobs, and arbitrates a queue of pending work.[30] Kubernetes has arrived alongside it, especially for serving, but it has not replaced it.
• Parallel file systems. Lustre, GPFS (IBM Storage Scale) and newer systems such as DAOS are still the storage layer under large machines; Aurora's DAOS system is rated at 31 TB/s.[6] AI training added new I/O patterns (millions of small files, huge checkpoints) but not a new category of storage.
• MPI. The MPI Forum approved MPI 5.0 on June 5, 2025.[29] NCCL and its peers borrowed MPI's collective vocabulary: all-reduce, all-gather, reduce-scatter and broadcast mean the same thing in a PyTorch training job as in a 1990s Fortran code.
• High-speed fabrics. InfiniBand and HPE Slingshot connect every system in the June 2026 top five except LineShine.[32] The scale-out fabric is still a fat tree or dragonfly of switches carrying RDMA traffic.
• The human side. Users still need accounts, allocations, training, documentation and someone to answer the ticket at 4:55 PM on a Friday.

Field note: Do not let the AI vocabulary intimidate your HPC staff. A "collective communication library" is MPI collectives on GPUs; a "scale-out fabric" is the InfiniBand network they already run; "checkpointing" is the same restart-file problem they have solved for twenty years. Map the new words onto the old skills and your team is most of the way there.

Roadmap of This Manual

The manual is organized in five parts. Read the Foundations and Hardware parts in order if you are building or buying; jump straight to Workloads if you are a user.

Key Takeaways

• AI and HPC now run on the same hardware: accelerators with HBM, a scale-up fabric, a scale-out fabric, a parallel file system, Linux and a scheduler. Four of the five exascale systems on the June 2026 TOP500 are accelerator machines.
• TOP500 (HPL, HPCG, HPL-MxP) and MLPerf measure different things. Use both to check vendor claims, then benchmark your own codes.
• Precision is a first-order design decision. Always ask which precision, dense or sparse, and per GPU or per rack when reading a FLOPS number.
• Native FP64 is no longer a given on AI-oriented GPUs; check the exact SKU. Mixed-precision solvers and Ozaki-style FP64 emulation can recover much of the gap for GEMM-heavy codes, but must be validated.
• Most scientific kernels and LLM token generation are memory-bound. The roofline model and arithmetic intensity explain why, and they should guide both purchases and user advice.
• The research computing center's user base and mission have broadened to include inference serving, data governance, hybrid capacity and liquid-cooled facilities.
• Linux, Slurm, MPI, parallel file systems and RDMA fabrics remain the foundation. Existing HPC skills carry over directly.

Sources

1. LineShine Debuts at No. 1 as the TOP500 Enters a New Global Exascale Era (TOP500, 2026)
2. June 2026 TOP500 list release page (TOP500, 2026)
3. ORNL's Frontier First to Break the Exaflop Ceiling (TOP500, 2022)
4. Frontier system specifications (Oak Ridge Leadership Computing Facility)
5. Using El Capitan Systems: Hardware Overview (Lawrence Livermore National Laboratory)
6. Aurora system specifications (Argonne Leadership Computing Facility)
7. NVIDIA Vera Rubin Delivers World-Class Supercomputers for Science (NVIDIA, 2026)
8. MLCommons Releases MLPerf Training v6.0 Results (MLCommons, 2026)
9. MLCommons Releases New MLPerf Inference v6.0 Benchmark Results (MLCommons, 2026)
10. Where the Industry Is Investing: A Look at MLPerf Inference v6.1 (MLCommons, 2026)
11. NVIDIA Ampere Architecture In-Depth (NVIDIA Technical Blog, 2020)
12. BFloat16: The secret to high performance on Cloud TPUs (Google Cloud, 2019)
13. Micikevicius et al., FP8 Formats for Deep Learning (arXiv, 2022)
14. AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm Standardize Next-Generation Narrow Precision Data Formats for AI (Open Compute Project, 2023)
15. Introducing NVFP4 for Efficient and Accurate Low-Precision Inference (NVIDIA Technical Blog, 2025)
16. Micikevicius et al., Mixed Precision Training (arXiv, 2017)
17. torch.set_float32_matmul_precision (PyTorch documentation)
18. Unlocking Tensor Core Performance with Floating Point Emulation in cuBLAS (NVIDIA Technical Blog, 2025)
19. DGEMM without FP64 Arithmetic: Using FP64 Emulation and FP8 Tensor Cores with Ozaki Scheme (arXiv, 2025)
20. HPL-MxP Benchmark: Mixed-Precision Algorithms, Iterative Refinement, and Scalable Data Generators (arXiv, 2025)
21. NVIDIA H100 Tensor Core GPU product specifications (NVIDIA)
22. NVIDIA HGX Platform specifications: HGX Rubin NVL8, B300, B200 (NVIDIA)
23. AMD Instinct MI355X GPU specifications (AMD)
24. Williams, Waterman and Patterson, Roofline: An Insightful Visual Performance Model (Communications of the ACM, 2009)
25. Gholami et al., AI and Memory Wall (arXiv, 2024)
26. JEDEC Releases JESD270-4 HBM4 Standard (JEDEC, 2025)
27. Energy Department Launches Genesis Mission (US Department of Energy, 2025)
28. Lam et al., GraphCast: Learning skillful medium-range global weather forecasting (arXiv / Science, 2023)
29. MPI Documents: MPI 5.0 (MPI Forum, 2025)
30. Slurm Workload Manager Overview (SchedMD)
31. NVIDIA NVLink and NVLink Switch specifications (NVIDIA)
32. June 2026 TOP500 list highlights (TOP500, 2026)