Infrastructure For Pcie And Infiniband Coupling Links - IBM z13s Technical Manual

Five types of fanout cards are supported by z13s servers. Each slot can hold one of the
following five fanouts:
PCIe Gen3 fanout card: This copper fanout provides connectivity to the PCIe switch card
in the PCIe I/O drawer.
ICA SR: This adapter provides 8x PCIe optical coupling connectivity between z13, z13s
servers, up to 150-meter (492 ft.) distance, 8 GB/s link rate.
Host Channel Adapter (HCA2-C): This copper fanout provides connectivity to the IFB-MP
card in the I/O drawer.
HCA3-O (12xIFB): This optical fanout provides 12x InfiniBand coupling link connectivity up
to 150-meter (492 ft.) distance to a z13, z13s, zEC12, zBC12, z196, and z114 servers.
HCA3-O LR (1xIFB): This optical long range fanout provides 1x InfiniBand coupling link
connectivity up to a 10 km (6.2 miles) unrepeated or 100 km (62 miles) when extended by
using z Systems qualified dense wavelength division multiplexing (DWDM) equipment
distance to z13, z13s, zEC12, zBC12, z196 and z114 servers.
The PCIe Gen3 fanout card comes with one port, the HCA3-O LR (1xIFB) fanout comes with
four ports, and other fanouts come with two ports.
Figure 4-6 illustrates the following PCIe and IFB connections from the CPC drawer:
PCIe Gen3 for connectivity to PCIe I/O drawer (not represented here, see Figure 4.4 on
page 143)
Another CPC through InfiniBand (either 12x or 1x HCA3-O)
Another z13s server connected through a dedicated PCIe ICA SR
Important: Figure 4-6 shows an I/O connection scheme that is not tied to a particular CPC
drawer. In a real configuration, I/O connectivity is spread across multiple CPC drawers, if
available, for I/O connection redundancy.
SMP
Cables
Figure 4-6 Infrastructure for PCIe and InfiniBand coupling links
148
IBM z13s Technical Guide
Drawer 1
Memory
PU PU
SC SC
PU PU
GX++
HCA3-O
PCIe Gen3
6 GB/s
Coupling Links:
12x InfiniBand
1x InfiniBand
Drawer 0
Memory
PU
SMP
Cables
SC
PU
FANOUTS PCIe Gen3
Coupling Links:
PCIe Gen3
PU
SC
SMP
Cables
PU
ICA SR
8 GB/s
PCIe Gen3 x8