D.18 Wiring For A Configuration Of Two Or Three C7000 Enclosures With Double-Density Server Blades Using The Hp Blc 4X Ddr Infiniband Gen2 Switch And External Infiniband Switches - HP Cluster Platform Express v2010 Overview

From
IB 24 Port Switch 2
Port 20
IB 24 Port Switch 2
Port 21
IB 24 Port Switch 2
Port 22
IB 24 Port Switch 2
Port 23
IB 24 Port Switch 2
Port 24
D.18 Wiring for a Configuration of Two or Three c7000 Enclosures with
Double-Density Server Blades Using the HP BLc 4X DDR InfiniBand Gen2
Switch and External InfiniBand Switches
Configurations with either two or three c7000 enclosures and double-density server blades that
use the HP BLc 4X DDR InfiniBand Gen2 switch require four external InfiniBand switches to
create a fully connected fabric. The wiring for two enclosures can be derived from the three
enclosure wiring tables by omitting the wiring for Enclosure 3.
The connection to Port 4 on the fourth external InfiniBand switch (IB-SW4) depends on whether
the configuration includes an external control node connected to the InfiniBand fabric or not. If
there is a control node connected to the InfiniBand fabric, the control node's HCA is connected
to that port. Otherwise, Port 4 on the fourth InfiniBand switch is connected to Port 8 on the
InfiniBand switch installed in Interconnect Module Bay 5/6 installed in Enclosure 1.
Wiring Table for the First InfiniBand Switch (IB-SW1)
From
IB Switch 1 24 Port
Switch Port 1
IB Switch 1 24 Port
Switch Port 2
IB Switch 1 24 Port
Switch Port 3
IB Switch 1 24 Port
Switch Port 4
IB Switch 1 24 Port
Switch Port 5
IB Switch 1 24 Port
Switch Port 6
IB Switch 1 24 Port
Switch Port 7
IB Switch 1 24 Port
Switch Port 8
IB Switch 1 24 Port
Switch Port 9
IB Switch 1 24 Port
Switch Port 10
D.18 Wiring for a Configuration of Two or Three c7000 Enclosures with Double-Density Server Blades Using the HP BLc
Label To
IB-SW2-P20
IB-SW2-P21
IB-SW2-P22
IB-SW2-P23
Control node InfiniBand
IB-SW2-P24
HCA
Label To
Enclosure 1 Interconnect
IB-SW1-P1 Module 5/6 Port 1
Enclosure 1 Interconnect
IB-SW1-P2 Module 5/6 Port 2
Enclosure 1 Interconnect
IB-SW1-P3 Module 5/6 Port 3
Enclosure 1 Interconnect
IB-SW1-P4 Module 5/6 Port 4
Enclosure 1 Interconnect
IB-SW1-P5 Module 7/8 Port 1
Enclosure 1 Interconnect
IB-SW1-P6 Module 7/8 Port 2
Enclosure 1 Interconnect
IB-SW1-P7 Module 7/8 Port 3
Enclosure 1 Interconnect
IB-SW1-P8 Module 7/8 Port 4
Enclosure 2 Interconnect
IB-SW1-P9 Module 5/6 Port 1
Enclosure 2 Interconnect
IB-SW1-P10 Module 5/6 Port 2
4X DDR InfiniBand Gen2 Switch and External InfiniBand Switches
Label
N/A N/A
N/A N/A
N/A N/A
N/A N/A
Control-P1
Label
E1-IMB5-P1
E1-IMB5-P2
E1-IMB5-P3
E1-IMB5-P4
E1-IMB7-P1
E1-IMB7-P2
E1-IMB7-P3
E1-IMB7-P4
E2-IMB5-P1
E2-IMB5-P2
Comment
Only used if the control node is
a rack-mount server connected
to the InfiniBand fabric
Comment
105