Introduction To The Infiniband And Pcie For I/O Infrastructure; Infiniband I/O Infrastructure; Pcie I/O Infrastructure - IBM z13s Technical Manual

4.1 Introduction to the InfiniBand and PCIe for I/O
infrastructure
z13s servers support two types of internal I/O infrastructure:
InfiniBand infrastructure for I/O drawers
Peripheral Component Interconnect Express (PCIe)-based infrastructure for PCIe
I/O drawers

4.1.1 InfiniBand I/O infrastructure

The InfiniBand I/O infrastructure was first made available on System z10 and is supported on
z13s servers. It consists of the following components:
InfiniBand fanouts that support the current 6 GBps InfiniBand I/O interconnect
InfiniBand I/O card domain multiplexers with redundant I/O interconnect to a 5U, eight-slot,
and two-domain I/O drawer.

4.1.2 PCIe I/O infrastructure

IBM continues the use of industry standards on the z Systems platform by offering a
Peripheral Component Interconnect Express Generation 3 (PCIe Gen3) I/O infrastructure.
The PCIe I/O infrastructure that is provided by the central processor complex (CPC) improves
I/O capability and flexibility, while allowing for the future integration of PCIe adapters and
accelerators.
The z13s PCIe I/O infrastructure consists of the following components:
PCIe fanouts that support 16 GBps I/O bus interconnection for CPC drawer connectivity to
the PCIe I/O drawers
The 7U, 32-slot, and 4-domain PCIe I/O drawer for PCIe I/O features
The z13s PCIe I/O infrastructure provides these benefits:
Increased bandwidth from the CPC drawer to the I/O domain in the PCIe I/O drawer
through a 16 GBps bus.
The PCIe I/O drawer supports double the number of I/O ports compared to an I/O drawer.
Up to 64 channels (32 PCIe I/O features) are supported versus the 32 channels (8 I/O
features) that are offered with the I/O drawer.
Better granularity for the storage area network (SAN) and the local area network (LAN):
For Fibre Channel connection (FICON), High Performance FICON on z Systems (zHPF),
and Fibre Channel Protocol (FCP) storage area networks, the FICON Express16S has
two channels per feature. For LAN connectivity, the Open System Adapter
(OSA)-Express5S GbE and the OSA-Express5S 1000BASE-T features have two ports
each, and the OSA-Express5S 10 GbE features have one port each.
Newly designed native PCIe features can be plugged into the PCIe I/O drawer, such as
Flash Express, zEnterprise Data Compression (zEDC) Express, and 10 GbE Remote
Direct Memory Access over Converged Ethernet (RoCE) Express.
138
IBM z13s Technical Guide