The General Absorbs the Specific: Why the CPU+GPU Tandem Survives the 'Do We Still Need GPUs?' Question

In June 2026 a machine with no GPUs reached the top of the TOP500. China's LineShine posted 2.198 sustained exaflops of double-precision HPL — the first CPU-only system past two sustained FP64 exaflops — on roughly 13.8 million custom Armv9 "LX2" cores (304-core sockets, SVE2 + SME), built in no small part to route around US GPU export controls. A GPU-less system at #1 is what put an old question back on the table, out loud.

Then three of the most credible names in the field — Jack Dongarra, Satoshi Matsuoka, and Torsten Hoefler, who were on opposite sides of the earlier FP64-vs-low-precision argument — published a joint paper, "Do We Still Need GPUs? Rethinking AI and Scientific Computing on Matrix-Enhanced CPUs." HPCwire's coverage distilled the mood to a phrase: "the general always absorbs the specific."

This post does two things: breaks down what the paper actually claims (it is far more modest than the headline), and argues why the CPU+GPU tandem survives it — not out of nostalgia, but because it is the optimal policy when your workload spans two bottleneck regimes at once.

What the paper actually says

The core claim is a claim about absorption, not victory:

A CPU with wide vectors (SVE/AVX), a matrix engine (SME/AMX), on-package HBM, and native precisions from FP64 down to FP4 "is no longer a conventional CPU. It is a general-purpose processor with accelerator-class numerical machinery."

GPUs originally filled two gaps: CPUs lacked parallel arithmetic and memory bandwidth. Put those on the CPU die and the historical justification for a separate device weakens. So the authors reframe the question — not "do we need GPUs?" but "what architectural features do we need?" Their list: bandwidth, dense linear algebra, precision options, mature compilers, usable programming models. Provide those, they argue, and the hardware label becomes secondary.

Their evidence leans hard on the fact that LLM inference is two workloads, not one:

with Fugaku (A64FX) as the existence proof — it led Top500, HPCG, and Graph500 simultaneously in 2020, all on CPUs.

Read the concessions, not the headline

The paper is careful in a way the headline isn't:

• GPUs "will remain vital for frontier-scale AI training" and compute-bound dense prefill.
• "FP8/FP4 is all you need" is "a research hypothesis, not an established result" — it fits dense matmul, not sparse / irregular / communication-heavy work. On the LX2, native FP64 is still the right path outside dense matrix multiply.
• An all-CPU fleet draws ~1.75–2.7× the per-user power of the newest HBM3e GPUs today.
• Software maturity is "the largest non-hardware gap" — CUDA/cuBLAS/cuDNN have two decades of tuning; CPU matrix-engine toolchains do not.
• LineShine runs ~0.8% of peak on HPCG — memory-bound reality still bites everyone.

Put those together and the paper is not arguing the GPU is useless. It is arguing: stop assuming a discrete accelerator is mandatory; ask what features the work needs. That is a convergence thesis. And convergence is the opposite of elimination.

Why the tandem survives — the don't-throw-anything-away argument

1. The paper's own map is a division of labor. CPU wins memory-bound decode, sparse, scalar, branchy work; GPU wins compute-bound prefill and dense training. When your analysis says "this substrate for these kernels, that one for those," you have written a routing table, not declared a winner.

2. Roofline forces both — inside a single workload. The two phases of inference sit on opposite sides of the ridge point. You cannot pick the "right" hardware for a job that is bandwidth-bound half the time and compute-bound the other half; you route each phase to the substrate that fits.

3. "The general absorbs the specific" points at convergence, not deletion. Putting AMX/SME on the die does not remove the accelerator — it integrates it; the CPU becomes internally heterogeneous. And the same convergence is happening from the other direction: Grace-Hopper / GB200 fuse a CPU and GPU under coherent memory. Both roads end at the superchip — the tandem getting tighter, not one partner leaving the board.

4. Amdahl keeps the CPU; the 10× keeps the GPU. Real scientific and agentic pipelines are full of orchestration, I/O, graph traversal, sparse solves, and scalar glue that GPUs are bad at — a GPU node has always needed a strong host CPU. That was never in doubt. What the "no GPU" reading discards is the one thing GPUs are ~10× at: dense training and compute-bound prefill. Keep the part that owns the tail (CPU) and the part that owns the peak (GPU).

5. Don't-throw-anything-away is also the economics and the strategy. Existing GPU fleets are enormous sunk capital; two decades of MPI/OpenMP + CUDA are enormous sunk software. The paper's own "software gap" admission means the CPU-matrix stack isn't ready to inherit the GPU's workload yet — so you can't discard it responsibly even if you wanted to. And LineShine's real driver is export-ban circumvention — a constraint-driven design, not proof CPUs are preferable. Where GPUs are available, using both is the rational move. Even the precision story is "keep everything": one die spanning FP64→FP4 lets you use the cheapest precision that is still correct per kernel, and fall back to FP64 where emulation fails.

6. This is multigrid, in hardware. It is the same smooth-cheap, escalate-the-residual policy as the V-cycle of models: use the inexpensive tier for everything it can handle, send only what it cannot to the expensive one. Heterogeneous compute is that idea in silicon — the CPU as the smoother (bandwidth-bound, sparse, scalar) and the GPU as the heavy coarse-grid solver (dense, compute-bound). A V-cycle never picks one level of the hierarchy and throws the rest away; the whole point is that every level does the work it is best at. "Don't throw anything away" is not sentiment — it is the optimal schedule when the workload spans multiple bottleneck regimes.

Keeping the strong claim honest

• The 48:1 node ratio is a decode-only, favorable slice: 48 CPU nodes of footprint, interconnect, and failure surface to match one GPU node's decode. The tandem just uses the one GPU node.
• The 1.75–2.7× power gap is real today, even discounting the HBM2-vs-HBM3e generation.
• HPCG at 0.8% of peak shows memory-bound reality caps everyone — an argument for balance, not for CPU-only.

A worked example: PENNANT, CPU → GPU → both

This isn't hypothetical. The practice's portable GPU methodology study took PENNANT — the unstructured-mesh Lagrangian hydrodynamics mini-app from the CORAL-2 suite that vetted El Capitan — across NVIDIA T4/H100 and AMD MI300X from one clean-room codebase, and the results trace exactly this arc.

Stage 1 — CPU-good. PENNANT began life as a CPU code (C++, OpenMP + MPI). And below a problem-size crossover around ~2M zones, a good multi-core CPU still wins:

Below the crossover the GPU is overhead-limited — launch and transfer costs dominate — and the CPU is simply the right tool.

Stage 2 — GPU-good, CPU-bad… with a catch. Ported to CUDA/HIP and pushed past the crossover, the GPU pulls ahead (a 38.5× headline — honestly caveated as vs a 1-thread cloud CPU). But two things complicate "GPU-good":

• The naive port was worse than the CPU: the first GPU kernels ran at 97% idle, ~200 s — slower than the 126 s CPU baseline (<1×). A partial OpenMP-target offload of only the cheap kernels ran 38% slower than single-thread CPU. A thoughtless port is CPU-bad and GPU-bad.
• Even fully optimized, the GPU leaves 97–99.7% of its compute idle on memory — identical on H100 and MI300X. The runtime-dominating kernels are "HBM-latency stall machines." So the GPU's advantage here is bandwidth, not compute — which is precisely the regime the roofline figure above says belongs to a CPU with HBM. "GPU-good" is really "GPU-HBM-good," and that qualifier is the whole ballgame.

Stage 3 — CPU+GPU good. The study's Phase 7 added a hybrid CPU/GPU dispatch flag that routes individual kernels to whichever substrate is faster. On the H100 PCIe, sending the SETVELDIFFNORM kernel to the CPU is the preferred build (−21%) — the host computation hides device latency the GPU can't. And the flag flips sign between vendors (−21% on H100, +1.3% on MI300X), so you measure per-target rather than assume. The hardware endpoint of this arc already exists: the AMD MI300A APU — CPU and GPU sharing one pool of HBM — the tandem cast into a single package.

So the honest resolution wasn't "port it to the GPU" or "keep it on the CPU." It was route each kernel to the substrate that fits its roofline, on the target you actually have — the don't-throw-anything-away policy, measured.

Bottom line

The paper is a healthy correction to GPU-by-default, and its convergence thesis is right: the accelerator is migrating onto the CPU die (and the CPU is migrating onto the GPU board). But "migrating onto" is the opposite of "no longer needed." The correct posture is the don't-throw-anything-away one: route each kernel to the substrate that fits its roofline, keep the mature GPU stack while the CPU-matrix software catches up, and treat the CPU+GPU pair as one heterogeneous machine — increasingly one superchip — rather than a contest with a winner.

Related: A V-Cycle of Models (the same route-by-fit idea, one layer up) · the LLM parallelism glossary · KV cache, the binding constraint (why decode is memory-bound) · the one-line order-of-magnitude check.