Latency Hiding & Occupancy: Interview Drills

Interview drills

Interview Drills for Latency Hiding & Occupancy centers on memory-latency cover ratio, long scoreboard stall %, and active warp depth. The objective is to connect profiler evidence to root-cause mechanism and release-safe action.

PROMPT
You see memory-latency cover ratio, long scoreboard stall %, and active warp depth on Latency Hiding & Occupancy. Walk through root cause and release decision.

STRONG ANSWER
1. Names failing workload/scene and first broken metric.
2. Explains Warp-level multithreading overlaps stalled warps with ready warps; occupancy and scheduler quality determine how much latency can be hidden.
3. Requests latency cover model, occupancy-vs-throughput curve, and stall reason timeline.
4. Proposes bounded fix + owner + validation matrix.

WEAK ANSWER
Suggests generic tuning without SIMT, warp, cache, or interconnect evidence.

Whiteboard diagram

Eligible warps vs long stalls

WARP SCHEDULER VIEW — Latency Hiding & Occupancy

cycle ->      0    1    2    3    4
eligible   [W1,W2,W5] [W2] [W2,W7] [W7] [W3,W7]
issued         W1      W2    W7      W7    W3
stall reason    -    dep wait  -   mem wait  -

Scheduler objective: keep issue slots non-empty.
Focus: demonstrate occupancy thresholds needed to hide DRAM latency

Debug tree to narrate

ROOT-CAUSE TREE — Latency Hiding & Occupancy

memory-latency cover ratio, long scoreboard stall %, and active warp depth regressed
        |
  reproducible on replay?
      /              \
    no                yes
    |                  |
env/test noise    counter triage
                   |
             compute-bound or memory-bound?
                /                  \
             compute            memory/interconnect
             issue stalls       cache/NoC/DRAM stalls

Stop at first failing mechanism, then patch.

GPU deep dive

Warp scheduling quality determines whether latency hiding survives real control-flow and memory variance.

Concept diagram

WARP SCHEDULING LOOP

ready warp? -> issue -> dependency wait -> reconverge -> issue

Metric graph

STALL REASON SHARE

long scoreboard    ███████
divergence replay  █████
barrier wait       ███

Reports and artifacts

• eligible warp ratio

• stall reason histogram

• barrier wait cycles

• scheduler fairness report

Mini case study

A barrier-heavy kernel looked occupancy-safe, but warp arrival imbalance turned sync points into dominant stalls.

Debug branches

• Compare scheduler policy traces under bursty workloads

• Measure reconvergence delay and predication side effects

• Quantify barrier idle time before tuning launch size

Senior review question

Ask: which metric and benchmark pairing proves this topic is truly closed in production context?

Key takeaways

• Always pair micro-kernel metrics with end-to-end workload impact.

• Lock toolchain, driver, and launch metadata before comparing performance results.

Common pitfalls

• Optimizing occupancy without checking memory-system saturation.

• Comparing profiler captures from different driver or compiler builds.

• Declaring wins without reproducible accuracy and performance gates.

Interview answer expansion

A strong interview answer for Latency Hiding & Occupancy starts with the workload and metric, then states the mechanism in plain language: Warp-level multithreading overlaps stalled warps with ready warps; occupancy and scheduler quality determine how much latency can be hidden.

Then it gives a measurement plan. Good answers name lane masks, issue slots, cache/transaction counters, memory-controller state, NoC congestion, thermal/DVFS telemetry, or stage queues depending on the topic.

Finally, it proposes one bounded fix and explains regression risk. GPU interviews reward tradeoff ownership: what improves, what may regress, and how you would know before tapeout or release.