PCIe Gen5: A pathway to address Data Explosion and Emerging Technologies

As the technology scales or shrinks, there are always some bottlenecks that need to be addressed sometimes it is the compute, sometimes it is the memory and in some cases, it is the limitations in existing protocols. We have seen PCIe bandwidth literally double every few years but we have had a critical need to move from PCI Gen 4 to Gen 5, the fastest we have to jump generations compared to previous generation jumps.

Some of the technology trends that are driving the need for PCIe Gen 5 are:

1. Cloud Computing: With everything moving to data, the networking protocol, Ethernet has had to scale up to meet the industry requirements and we have seen the scaling of towards 800GB Ethernet. With 400GB Ethernet, we are looking at around 100 GT/s bandwidth on PCIe.
2. Artificial Intelligence: AI has re-emerged thanks to today’s compute and storage capacity improvements. When doing AI/ML/DL, we are talking of massive amounts of training data and graphics data that needs to be processed quickly and efficiently, so this processing is being offloaded to specialized hardware or accelerators. This back and forth communication between a CPU and accelerators is over PCIe.
3. The emergence of new protocols leveraging the existing PCIe infrastructure like NVMe based SSDs for storage and the Cache Coherent Interconnect for Accelerators (CCIX) or Compute Express Link (CXL) to enable cache coherency between CPUs and Accelerators.

While there is an enormous need for higher bandwidth, there is also a requirement of using existing infrastructure like PCIe form factors, backward compatibility and lowering power dissipation. PCIe Gen 5 was born out of the need for doubling the bandwidth from Gen4 while keeping the same form factor and achieving these in as quick a jump between generations.

PCIe Gen5 delivers a raw bit rate/lane of 32 GT/s (as compared to 16GT/s of Gen4) and this means that a x16 link would be able to achieve a bandwidth of ~128BG/s (32GT/s x 32 lanes/8 bits per byte x 128/130 encoding x 2 for duplex connection) and with a x32 link we would be able to achieve ~256GB/s meeting the bandwidth requirements of 400GB & 800GB Ethernet.

To achieve the above requirement for PCIe Gen5, there were no changes that were done to the Link and the Transaction layer. The Physical layer was the focus for making these improvements. Some of the key updates are:

• Equalization & new precoding mechanism
• EIEOS, SDS, SKIP & control SKIP ordered sets
• Support for Alternate Protocols
  • Via modified TS1/TS2 ordered sets

The support for Alternate Protocols is a key addition as it enables the future emerging protocols to leverage some of the PCIe infrastructures like Compute Express Link (CXL). You can check out the additional list of PCIe Gen 5 features here.

If you are an existing user of Mentor’s PCIe 4.0 VIP, you can seamlessly move to PCIe 5.0 QVIP, with the below modifications to your existing testbench environment:

1. Connect your PCIe Gen5 compliant DUT with PCIe QVIP
2. Modify Agent descriptor for PCIe version, speed and interface type to Gen5
3. Speed change to Gen5
   •  For autonomous speed change to max speed i.e. Gen5 in agent descriptor
   • For initiating speed change from testbench using a sequence

The details on moving PCIe Gen4 QVIP to Gen5 QVIP can be found here.

I hope everyone is staying safe and I hope to put out a few more blogs out soon.