PCIe/CXL Deep Dive · All levels
PCIe Protocol Stack: Tricky Q&A
Senior interview and review questions for PCIe Protocol Stack.
Section Q&A bank
Use these drills after completing all topics in PCIe Protocol Stack. Answer with workload context, mechanism proof, artifact, owner, and release decision.
Why can a TLP arrive at the receiver while the link is still in recovery?
diagram
[INT][PCIE-CXL][PROTOCOL-STACK]
Q: Why can a TLP arrive at the receiver while the link is still in recovery?
A:
Upper layers may retain queued TLPs while LTSSM re-trains, but the data link layer must block new transmission until the link returns to L0 and replay state is synchronized. In-flight TLPs may be replayed after recovery; ordering and completion semantics depend on whether the transition preserved DL state.
FOLLOW-UP TRAP: Assuming transaction layer can continue sending through any LTSSM state without DL gating.What is the practical difference between posted and non-posted TLP flow control?
diagram
[INT][PCIE-CXL][PROTOCOL-STACK]
Q: What is the practical difference between posted and non-posted TLP flow control?
A:
Posted writes consume receiver buffer credits immediately and do not require completions, so credit accounting errors stall write traffic silently. Non-posted requests require completion buffers and tag matching; starvation shows up as completion timeouts. Debug must separate credit exhaustion from ordering blocks.
FOLLOW-UP TRAP: Treating all TLP stalls as bandwidth limits without checking buffer credit pools.How do PCIe ordering rules interact with multi-function devices?
diagram
[INT][PCIE-CXL][PROTOCOL-STACK]
Q: How do PCIe ordering rules interact with multi-function devices?
A:
Ordering is defined relative to traffic class and path, not just function number. Different functions on the same device may share switch ports and VC resources, so a low-priority function can still affect ordering-visible behavior if it shares buffer pools or causes credit backpressure.
FOLLOW-UP TRAP: Assuming strict per-function isolation without checking shared bridge resources.When does ECRC matter if LCRC already protects the link?
diagram
[INT][PCIE-CXL][PROTOCOL-STACK]
Q: When does ECRC matter if LCRC already protects the link?
A:
LCRC protects symbol delivery across a single link segment; ECRC protects end-to-end TLP payload integrity through switches and retimers. A switch corruption or soft logic bug can pass LCRC checks hop-by-hop while failing ECRC at the endpoint, which is critical for RAS and poison handling.
FOLLOW-UP TRAP: Disabling ECRC because the link BER is low.Q&A drill guide
diagram
WORKLOAD -> PCIe/CXL SYMPTOM -> TIMING/QUEUE METRIC -> ROOT CAUSE -> FIX -> REGRESSIONSketch while answering
diagram
PROTOCOL STACK FLOW
App -> TLP (TL) -> DLLP/seq (DL) -> symbols (PHY) -> link partnerKey takeaways
Always tie controller and PHY counter shifts to application latency and throughput outcomes.
Lock firmware timing profile, thermal condition, and DIMM state before comparing PCIe/CXL captures.
Common pitfalls
Chasing peak bandwidth while ignoring p99 latency and fairness tails.
Changing timing guardbands without separating SI noise from scheduling issues.
Declaring closure without reliability gates, fault injection, and regression replay.