Infiniband Host Channel Adapter - IBM Power 720 Overview

For more information about FCoE, see An Introduction to Fibre Channel over Ethernet, and
Fibre Channel over Convergence Enhanced Ethernet, REDP-4493.

2.7.10 InfiniBand Host Channel adapter

The InfiniBand Architecture (IBA) is an industry-standard architecture for server I/O and
inter-server communication. It was developed by the InfiniBand Trade Association (IBTA) to
provide the levels of reliability, availability, performance, and scalability necessary for present
and future server systems with levels significantly better than can be achieved using
bus-oriented I/O structures.
InfiniBand (IB) is an open set of interconnect standards and specifications. The main IB
specification has been published by the InfiniBand Trade Association and is available at:
http://www.infinibandta.org/
InfiniBand is based on a switched fabric architecture of serial point-to-point links, where these
IB links can be connected to either host channel adapters (HCAs), used primarily in servers,
or target channel adapters (TCAs), used primarily in storage subsystems.
The InfiniBand physical connection consists of multiple byte lanes. Each individual byte lane
is a four-wire, 2.5, 5.0, or 10.0 Gbps bidirectional connection. Combinations of link width and
byte-lane speed allow for overall link speeds from 2.5 Gbps to 120 Gbps. The architecture
defines a layered hardware protocol and a software layer to manage initialization and the
communication between devices. Each link can support multiple transport services for
reliability and multiple prioritized virtual communication channels.
IBM offers the GX++ 12X DDR Adapter (FC EJ04) that plugs into the system backplane
(GX++ slot). One GX++ slot is available on the Power 720. One or two GX++ slots are
available on the Power 740, if used with one or two processor cards. Detailed information can
be found in 2.5, "System bus" on page 61.
By attaching a 12X to 4X converter cable (FC 1828, FC 1841, or FC 1842) to FC EJ04, a
supported IB switch can be attached. AIX, IBM i, and Linux operating systems are supported.
A new PCIe Gen2 LP 2-Port 4X InfiniBand quad data rates (QDR) 40 Gb adapter (FC 5283)
is available. The PCIe Gen2 low-profile adapter provides two high-speed 4X InfiniBand
connections for IP over IB usage in the Power 720 and Power 740. On the Power 720 and
Power 740, this adapter is supported in PCIe Gen2 slots. The following types of QDR IB
cables are provided for attachment to the QDR adapter and its Quad Small Form-Factor
Pluggable (QSFP) connectors:
Copper cables provide 1-meter, 3-meter, and 5-meter lengths (FC 3287, FC 3288, and
FC 3289).
Optical cables provide 10-meter and 30-meter lengths (FC 3290 and FC 3293). These are
QSFP/QSFP cables that also attach to QSFP ports on the switch.
The FC 5283 QDR adapter attaches to the QLogic QDR switches. These switches can be
ordered from IBM by using the following machine type and model numbering:
7874-036 is a QLogic 12200 36-port, 40 Gbps InfiniBand Switch that cost-effectively links
workgroup resources into a cluster.
7874-072 is a QLogic 12800-040 72-port, 40 Gbps InfiniBand switch that links resources
using a scalable, low-latency fabric, supporting up to four 18-port QDR Leaf Modules.
7874-324 is a QLogic 12800-180 324-port 40 Gbps InfiniBand switch designed to maintain
larger clusters, supporting up to eighteen 18-port QDR Leaf Modules.
Chapter 2. Architecture and technical overview
73