PCIe/CXL Deep Dive · All levels
PCIe Transactions and DMA: Tricky Q&A
Senior interview and review questions for PCIe Transactions and DMA.
Section Q&A bank
Use these drills after completing all topics in PCIe Transactions and DMA. Answer with workload context, mechanism proof, artifact, owner, and release decision.
Why do completion timeouts sometimes indicate tag leaks rather than link loss?
diagram
[INT][PCIE-CXL][PCIE-TRANSACTIONS]
Q: Why do completion timeouts sometimes indicate tag leaks rather than link loss?
A:
If a request tag is not freed because a completion was dropped or mis-associated, the requester stops issuing new non-posted transactions when the tag pool exhausts. Link may remain in L0 with good BER while performance collapses.
FOLLOW-UP TRAP: Resetting the link on timeout without checking outstanding tag counters.How do relaxed ordering attributes change debug strategy?
diagram
[INT][PCIE-CXL][PCIE-TRANSACTIONS]
Q: How do relaxed ordering attributes change debug strategy?
A:
Reordered traffic can pass functional tests while violating system assumptions. Debug must use memory barriers, ordered traffic classes, or explicit fences and compare against a golden sequential model.
FOLLOW-UP TRAP: Debugging coherency bugs without forcing strongly ordered traffic as a baseline.What prevents P2P even when two GPUs share a switch?
diagram
[INT][PCIE-CXL][PCIE-TRANSACTIONS]
Q: What prevents P2P even when two GPUs share a switch?
A:
ACS redirect, IOMMU policy, missing P2P BAR overlap, or driver fallback can force host staging. Verify switch P2P support, enable bits, and address windows before optimizing zero-copy paths.
FOLLOW-UP TRAP: Assuming same-switch placement guarantees P2P without ACS and IOMMU checks.When are atomic TLPs preferred over locked cycles or CPU fences?
diagram
[INT][PCIE-CXL][PCIE-TRANSACTIONS]
Q: When are atomic TLPs preferred over locked cycles or CPU fences?
A:
Atomics provide defined semantics for accelerator offload without bus locking. They are preferred for device-initiated counters and lockless structures when coherency domain includes the endpoint and ordering rules are honored.
FOLLOW-UP TRAP: Using atomics across non-coherent paths without CXL.cache or equivalent support.Q&A drill guide
diagram
WORKLOAD -> PCIe/CXL SYMPTOM -> TIMING/QUEUE METRIC -> ROOT CAUSE -> FIX -> REGRESSIONSketch while answering
diagram
TRANSACTION LIFECYCLE
MemRd -> tag alloc -> completion(s) -> tag freeKey takeaways
Always tie controller and PHY counter shifts to application latency and throughput outcomes.
Lock firmware timing profile, thermal condition, and DIMM state before comparing PCIe/CXL captures.
Common pitfalls
Chasing peak bandwidth while ignoring p99 latency and fairness tails.
Changing timing guardbands without separating SI noise from scheduling issues.
Declaring closure without reliability gates, fault injection, and regression replay.