Bigger Isn't Better: Finding the Energy-Optimal Cache with gem5 and McPAT

Motivation

A recurring question in system design is deceptively simple: how much cache is enough? More cache reduces misses and speeds up memory-bound code — but it also costs area, static (leakage) power, and money. The honest answer requires a model that can predict both performance and energy as we change the memory hierarchy.

This is exactly the kind of "what-if" that cycle-level simulation answers and that functional emulation cannot. It's worth being precise about the distinction:

• QEMU runs software fast via dynamic binary translation, but it has no timing model — no cycles, no caches, no memory latency. Change the cache size and QEMU produces the same result at the same (unmodeled) speed.
• gem5 models the microarchitecture — pipeline, caches, memory controllers, DRAM timing — advancing a simulated clock cycle by cycle. It is orders of magnitude slower, but it predicts performance and explains it.

For power, we pair gem5 with McPAT, which turns activity statistics into power and area estimates. We target the ARM (aarch64) ISA throughout, since it is the architecture of modern server and edge SoCs.

Experimental setup

Everything below runs in a native arm64 environment so the simulator itself is fast (the target ISA gem5 simulates is independent of the host).

• Simulator: gem5 v25.1.0.1, ARM target, built natively on arm64.
• CPU model: in-order timing core (ArmTimingSimpleCPU) at 1 GHz, syscall-emulation mode.
• Memory hierarchy: 32 KiB L1I + 32 KiB L1D (2-way), a shared L2 we sweep, and a DDR3-1600 controller.
• Power model: McPAT 1.3, fed by gem5's per-run statistics.

The workload: pointer chasing

To make cache size matter, the workload's working set must sit between "small cache" and "large cache." We use a pointer-chasing microbenchmark: a single randomized permutation cycle over a fixed-size buffer, traversed with idx = a[idx]. Each access depends on the previous one, so the CPU cannot overlap misses and hardware prefetchers cannot predict the stream. Runtime is therefore dominated by the latency of whichever level of the hierarchy actually holds the working set.

/* One big random cycle through the buffer, then chase it. */
for (size_t i = 0; i < n; i++) a[order[i]] = order[(i + 1) % n];
size_t idx = 0;
for (long k = 0; k < chases; k++) idx = a[idx];   /* dependent loads */

The working set is fixed at 512 KiB; we sweep the L2 size from 64 KiB to 4 MiB. The benchmark is deterministic, so the dynamic instruction count is identical across every run — only the timing changes. That invariant is our cleanliness check.

Result 1 — the performance knee

 runtime (ms)
 64KiB  149 |████████████████████████████████████████
128KiB  138 |█████████████████████████████████████
256KiB  113 |██████████████████████████████
512KiB   61 |████████████████   <- working set (512 KiB) now fits in L2
  1MiB   58 |███████████████
  2MiB   58 |███████████████
  4MiB   58 |███████████████
             lower = faster ->

The story is the sharp drop between 256 KiB and 512 KiB. While the 512 KiB working set does not fit in L2, accesses spill to DRAM (57–93% L2 miss rate) and each miss pays DRAM latency. Once L2 is large enough to hold the set, the miss rate collapses to a few percent, runtime halves, and IPC jumps roughly 2.5×. Beyond 512 KiB–1 MiB there is no further benefit — the residual ~1.6% misses are compulsory (cold) misses you cannot cache away.

Two invariants confirm the experiment is clean: the committed instruction count is identical across all runs (only timing moved), and the L1D miss rate is constant (we isolated L2 as the only variable).

Result 2 — the energy sweet spot

Performance says "512 KiB is enough; bigger doesn't help." Energy tells the other half of the story. Feeding each run's statistics to McPAT and multiplying total power by runtime:

 total energy (mJ)
 64KiB  28.6 |█████████████████
128KiB  28.3 |█████████████████
256KiB  27.3 |████████████████
512KiB  18.9 |███████████        <- MINIMUM energy
  1MiB  25.3 |███████████████
  2MiB  38.6 |███████████████████████
  4MiB  66.5 |████████████████████████████████████████
              smaller <- L2 size -> bigger

The energy curve is U-shaped, with a minimum exactly at the working-set size:

• Below 512 KiB, the program runs long (DRAM-latency misses), so dynamic energy and accumulated leakage are both high.
• At 512 KiB, the working set just fits: runtime collapses while the cache is still small enough that leakage is modest. This is the energy minimum.
• Above 512 KiB, energy rises again. Performance has already plateaued, but each doubling of L2 roughly doubles its leakage power — more transistors sitting idle. You pay more static energy for zero speed benefit.

Underneath, two components pull in opposite directions: memory-controller power falls as L2 grows (fewer DRAM accesses), while cache leakage power rises. Their crossover creates the U.

The takeaway

Performance-optimal is not energy-optimal. The cache that maximizes speed is the smallest one that holds the working set; growing it further is diminishing returns on performance and negative returns on energy. A timing model alone would have stopped at "512 KiB is fast." Pairing it with a power model reveals that 512 KiB is also where energy bottoms out — and that overshooting is actively wasteful.

This is the value of system-level simulation: it lets you locate these tradeoffs before committing them to silicon, and it generalizes directly to questions like DRAM technology choice, multi-level cache balancing, and accelerator memory sizing.

Honest caveats

We are deliberate about which numbers to trust:

• The in-order CPU model does not emit out-of-order microarchitectural counters, so McPAT's core power is a calibrated approximation. It is, however, nearly constant across the sweep, so it does not distort the shape of either curve.
• Leakage here assumes 40 nm high-performance devices (intentionally leaky). A low-power device type would flatten the rise; the existence of the energy minimum is robust, its exact depth is technology-dependent.
• These are on-chip figures (core, caches, memory controller). DRAM-chip energy is modeled separately (DRAMPower) and would penalize the small-cache, high-traffic configurations even more — strengthening, not weakening, the conclusion.

Reproducibility

The full pipeline — gem5 build, the pointer-chase workload, the parameterized ARM config, the sweep driver, and the gem5→McPAT power flow — is scripted end to end and runs in a single container. Every figure above traces back to a stats.txt and a McPAT run. Reproducibility is the point: the same harness answers the next question (working-set sweep, DRAM technology, out-of-order cores) by changing one parameter.

References

gem5 follows a citation policy and emits a citations.bib for each run, listing the papers behind the specific models exercised. For this study those are the simulator itself, the version 20.0+ release, and the DRAM controller model; McPAT is cited separately.

1. N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, A. Saidi, A. Basu, J. Hestness, D. R. Hower, T. Krishna, S. Sardashti, R. Sen, K. Sewell, M. Shoaib, N. Vaish, M. D. Hill, and D. A. Wood. "The gem5 simulator." ACM SIGARCH Computer Architecture News, 39(2):1–7, 2011. doi:10.1145/2024716.2024718
2. J. Lowe-Power, A. M. Ahmad, A. Akram, M. Alian, R. Amslinger, M. Andreozzi, et al. "The gem5 Simulator: Version 20.0+." arXiv preprint arXiv:2007.03152, 2020. arxiv.org/abs/2007.03152
3. A. Hansson, N. Agarwal, A. Kolli, T. F. Wenisch, and A. N. Udipi. "Simulating DRAM controllers for future system architecture exploration." In 2014 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 201–210, 2014. doi:10.1109/ISPASS.2014.6844484
4. S. Li, J. H. Ahn, R. D. Strong, J. B. Brockman, D. M. Tullsen, and N. P. Jouppi. "McPAT: An Integrated Power, Area, and Timing Modeling Framework for Multicore and Manycore Architectures." In Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO-42), pp. 469–480, 2009. doi:10.1145/1669112.1669172