Infiniband Switches; Fibre Channel Switches - IBM NeXtScale System Planning And Implementation Manual

Feature code
10 Gb top-of-rack switches
A1M6
A4HP
These switches do not feature front (port side) to rear cooling air flow, so they are
not good candidates for racks that feature NeXtScale systems because of the
number of cables that must be routed from the front to the rear of the rack.

5.6.2 InfiniBand switches

The InfiniBand switches, which are listed in Table 5-13, are available as part of
the Intelligent Cluster offering. As with the Ethernet switches that require cooling
from the ports to the back of the switch, the InfiniBand switches need the same
air flow path. However, on the InfiniBand switches, it is referred to as opposite
port side exhaust (oPSE). It is recommend that the System Networking
Recessed 19-inch 4-Post Rail Kit is installed for cable bend radius reasons.
Table 5-13 InfiniBand switch feature codes
Feature code
A2EZ
A2Y7
6676
6925

5.6.3 Fibre Channel switches

For I/O intensive applications, 8 Gb and 16 Gb Fibre Channel networks, which
are compatible with 8 Gb and 4 Gb storage devices, are popular because of their
high I/Os per second (IOPS) capability and high reliability. In larger clusters or
systems with high I/O demands, there are several nodes that are connected to
SAN storage, which then share storage via a parallel file system (such as IBM's
GPFS) to the rest of the servers. Table 5-14 on page 146 lists the current IBM
16 Gb Fibre Channel switch offerings. These switches should be mounted facing
the rear of the rack because they all have rear to front (port side) cooling air flow.
The fiber optic connections must pass from the adapters that are installed in the
front of the NeXtScale nodes to the ports at the rear of the rack.
Description
Cisco 3750X-48T-L (48 port, Layer 2/3)
Edge-Core ECS5610-52S 10 GbE Switch
Description
Mellanox SX6036 FDR10 InfiniBand Switch (oPSE)
Mellanox SX6036 FDR14 InfiniBand Switch (oPSE)
Intel 12200 QDR IB Redundant Power Switch (oPSE)
Intel 12200 QDR InfiniBand Switch (oPSE)
Chapter 5. Rack planning
145