Fast Memory Isn't Fast Alone: Bandwidth, Parallelism, and the HBM Question

Where we left off

In our cache-size investigation we used a pointer-chasing benchmark to expose memory latency: performance was governed by whether the working set fit in cache (the L2 "knee"), and energy was minimized by the smallest cache that still fit. That study deliberately probed one axis of the memory system — capacity and latency.

This post probes the orthogonal axis: bandwidth. How fast can the system move bulk data, and does newer, higher-bandwidth DRAM actually deliver? The answer turns out to depend on something neither the datasheet nor the latency study would tell you.

Two benchmarks, two different questions

Pointer chasing and STREAM are often lumped together as "memory benchmarks," but they measure opposite things:

The key word is memory-level parallelism (MLP) — how many memory requests are outstanding at once. Latency is what you wait when requests are serialized; bandwidth is what you achieve when many overlap. This distinction drives everything below.

Setup

We simulate an ARM (aarch64) system in gem5 v25 and use two drivers:

• STREAM Triad on an out-of-order core (ArmO3CPU). The out-of-order core matters: an in-order core blocks on every load (one outstanding request) and cannot generate bandwidth no matter how fast the memory is. The O3 core keeps several requests in flight.
• gem5's traffic generator (PyTrafficGen), which has no CPU at all — it injects memory requests at a high rate with effectively unlimited outstanding, so the memory itself becomes the bottleneck and we see each DRAM's true capability.

We sweep five DRAM models from a single memory channel: DDR3-1600, DDR4-2400, DDR5-6400, LPDDR5-6400, and one HBM channel.

Result 1 — one core can't use fast memory

Running STREAM Triad on a single out-of-order core across the DRAM types:

The result is counterintuitive: peak bandwidth nearly doubles from DDR3 to DDR5, yet achieved bandwidth stays stuck at 3–6 GB/s — and the highest-bandwidth memories deliver the least. Two effects combine:

1. A single core cannot saturate modern DRAM. Achieved bandwidth is capped by how many requests the core keeps in flight, not by the memory's peak. DDR5/HBM bandwidth simply goes unused.
2. When you can't fill the bandwidth, latency decides. DDR5, HBM, and LPDDR5 trade latency away for bandwidth. A parallelism-limited core is latency-sensitive, so it actually prefers low-latency DDR3/DDR4 and does worst on the highest-bandwidth DDR5.

Result 2 — flood the memory, and bandwidth scales

Now we replace the core with the traffic generator, which removes the parallelism limit entirely:

 achieved bandwidth (GB/s) — now it tracks the technology
 DDR3-1600   |████████████████ 10.0
 LPDDR5-6400 |███████████████████ 12.0
 DDR4-2400   |██████████████████████ 13.8
 HBM 1-chan  |█████████████████████████ 15.2
 DDR5-6400   |██████████████████████████████████████ 23.5

With enough outstanding requests, achieved bandwidth climbs from 10 to 23.5 GB/s across the generations, and utilization rises to 69–91% of peak (versus 16–39% for the single core). Each technology finally shows what it can do.

The punchline: same memory, opposite verdict

Compare DDR3 against DDR5 under the two drivers:

Identical memory hardware, opposite conclusions — decided entirely by how much parallelism the consumer brings. A single core is starved and prefers low-latency DDR3; a massively parallel consumer unlocks DDR5's bandwidth advantage.

Why this is the HBM rationale

This is exactly why high-bandwidth memory (HBM) is paired with GPUs and many-core accelerators rather than single CPU cores. A GPU runs thousands of concurrent threads, generating the memory-level parallelism needed to fill HBM's enormous bandwidth. A single CPU core cannot, and would see little benefit — it might even prefer cheaper, low-latency DDR. The hardware pairing follows directly from the parallelism of the workload, and we can see it in the numbers without any silicon.

The complete picture

Across this study and the previous one, three lenses on the same memory system:

1. Latency (pointer chase): performance is governed by whether the working set fits in cache.
2. Energy (gem5 + McPAT): the energy-optimal cache is the smallest one that fits; leakage punishes overshoot.
3. Bandwidth (STREAM + traffic generator): high-bandwidth memory only pays off with enough parallelism to exploit it.

Each is a question a datasheet cannot answer and that functional emulation cannot model — but that cycle-level simulation can, cheaply and before committing anything to hardware.

Caveats

• Bandwidth figures are for a single memory channel per technology; real systems use multiple channels, scaling absolute numbers but not the qualitative story.
• The traffic generator is synthetic (uniform linear reads); it measures achievable bandwidth, an upper bound a real workload approaches but rarely reaches.
• These are syscall-emulation runs of a microbenchmark, chosen to isolate the effect cleanly rather than to model a full application.

References

These runs use gem5; per its citation policy, the relevant papers (emitted in each run's citations.bib) are the simulator, the version 20.0+ release, and the DRAM controller model.

1. N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, et al. "The gem5 simulator." ACM SIGARCH Computer Architecture News, 39(2):1–7, 2011. doi:10.1145/2024716.2024718
2. J. Lowe-Power, A. M. Ahmad, A. Akram, M. Alian, et al. "The gem5 Simulator: Version 20.0+." arXiv preprint arXiv:2007.03152, 2020. arxiv.org/abs/2007.03152
3. A. Hansson, N. Agarwal, A. Kolli, T. F. Wenisch, and A. N. Udipi. "Simulating DRAM controllers for future system architecture exploration." In 2014 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 201–210, 2014. doi:10.1109/ISPASS.2014.6844484