GPU Design · All levels
GPU Memory Hierarchy: Tricky Q&A
Senior interview and review questions for GPU Memory Hierarchy.
Section Q&A bank
Use these drills after completing all topics in GPU Memory Hierarchy. Answer with workload context, mechanism proof, artifact, owner, and release decision.
How do you prove coalescing, not cache capacity, is the limiting factor?
diagram
[INT][GPU][MEMORY-HIERARCHY]
Q: How do you prove coalescing, not cache capacity, is the limiting factor?
A:
Transaction-per-request inflation and wasted-byte metrics rising with stable cache hit behavior indicates access-pattern inefficiency.
FOLLOW-UP TRAP: Attributing every bandwidth issue to cache misses.Why can high L2 hit rate still coexist with poor throughput?
diagram
[INT][GPU][MEMORY-HIERARCHY]
Q: Why can high L2 hit rate still coexist with poor throughput?
A:
Contention, serialization, and replay overhead can keep effective bandwidth low even when nominal hit ratio appears strong.
FOLLOW-UP TRAP: Using hit rate as a standalone closure metric.What is the practical sign of shared-memory bank conflict dominance?
diagram
[INT][GPU][MEMORY-HIERARCHY]
Q: What is the practical sign of shared-memory bank conflict dominance?
A:
Kernel speedup tracks with layout/index remapping while global-memory counters remain mostly unchanged.
FOLLOW-UP TRAP: Tuning DRAM policies before fixing local bank access patterns.When should memory hierarchy tuning stop and algorithm refactor begin?
diagram
[INT][GPU][MEMORY-HIERARCHY]
Q: When should memory hierarchy tuning stop and algorithm refactor begin?
A:
When roofline position remains memory-bound after coalescing, tiling, and cache-policy improvements on representative workloads.
FOLLOW-UP TRAP: Repeating low-level tweaks after reaching structural limits.Q&A drill guide
diagram
WORKLOAD -> BOTTLENECK -> METRIC -> CHANGE -> REGRESSIONSketch while answering
diagram
MEMORY HIERARCHY
register -> shared/L1 -> L2/LLC -> HBM/GDDR
access pattern quality decides latencyKey takeaways
Always pair micro-kernel metrics with end-to-end workload impact.
Lock toolchain, driver, and launch metadata before comparing performance results.
Common pitfalls
Optimizing occupancy without checking memory-system saturation.
Comparing profiler captures from different driver or compiler builds.
Declaring wins without reproducible accuracy and performance gates.