Compute Fabric & Memory System

Section goal

On-chip fabric topology, memory controller behavior, host interface, and system-level bandwidth/latency bottlenecks.

How to study this section

1. Start with each topic hub and sketch the SIMT-to-silicon mechanism.

2. Use reports and debug pages to prove bottlenecks with evidence.

3. Practice worked examples and interview drills for design-review fluency.

4. Close with checklist and silicon impact before signoff claims.

Topics

1. crossbar-and-noc-topology/ - Crossbar & NoC Topology

2. memory-controllers-hbm-gddr/ - Memory Controllers (HBM/GDDR)

3. host-pcie-dma-interface/ - Host PCIe DMA Interface

4. bandwidth-latency-bottlenecks/ - Bandwidth & Latency Bottlenecks

Related topics

• GPU Design course hub

GPU deep dive

Fabric and memory-controller behavior decides scaling long before peak ALU utilization is reached.

Concept diagram

COMPUTE FABRIC

SM clusters <-> NoC <-> L2 <-> memory controllers <-> HBM

Metric graph

SCALING EFFICIENCY

single GPU        ███████████ 100%
with heavy NoC    ████████
with tuned QoS    █████████

Reports and artifacts

• NoC congestion map

• HBM controller queue stats

• PCIe/DMA overlap timeline

• roofline position report

Mini case study

Crossbar arbitration favored bulk traffic and starved latency-sensitive queues, collapsing tail performance.

Debug branches

• Track per-link hotspots, not only aggregate BW

• Inspect controller page-hit and queue depth behavior

• Validate host-device overlap during peak traffic windows

Senior review question

Ask: which metric and benchmark pairing proves this topic is truly closed in production context?