Memory Bandwidth and Latency · arch

Computer ArchitectureAll levels

5topics
5lessons

Cores stall on memory while DRAM looks idle. The gap between headline gigabytes per second and bytes that reach the core is lost across caches, the on-chip network and the controller.

Where gigabytes per second disappear on the way to DRAM.

One illustrative waterfall: 164 GB/s of core demand arrives as 118 GB/s of DRAM service, about 72 percent.

Lesson	What it covers
Peak, protocol and workload bandwidth	Three bandwidths and a worked efficiency decomposition.
Little's law, MLP and the waterfall	Queue sizing, memory-level parallelism and per-hop loss attribution.
Row buffers and controller scheduling	Row hits, bank conflicts, FR-FCFS, write drains and refresh.
Prefetch accuracy, coverage and pollution	What prefetching earns, what it wastes and what it evicts.
Counters, QoS and debug	Counter methodology, first-saturated-queue triage and guarded tuning.

Check yourself Core counters show rising back-end memory stalls while DRAM reports 72 percent utilization. Where is the missing bandwidth?

It is lost upstream of DRAM: LLC filtering, NoC congestion and controller queueing leave 46 GB/s behind, so DRAM never sees the full 164 GB/s of demand.

Up nextPeak, Protocol and Workload Bandwidth

Practice this lesson Previous lesson Report an error