50% Blocking Infiniband - IBM NeXtScale System Planning And Implementation Manual

5.7.2 50% blocking InfiniBand

Figure 5-20 shows a 50%-blocking (two node ports for every uplink port)
InfiniBand configuration. On the left side is a rack with six chassis for a total of 72
CPU nodes. Three 36-port InfiniBand switches and two 48-port 1 Gbps Ethernet
switches provide the network connectivity.
1G Ethernet Switch 2
42
IB Switch 3
41
40 11
9
39
7
38
5
37
3
36
1
35
34
11
33
9
32 7
5
31
3
30
1
29
Cable Channel 3
28
11
27
9
26 7
25 5
3
24
1
23
22
IB Switch 2
21
11
20
9
19
7
18
5
17
3
16
Cable Channel 1
1
15
14
11
13
9
12
7
5
11
3
10
1
9
8
11
7
9
6 7
5 5
3
4
1
3
IB Switch 1
2
1G Ethernet Switch 1
1
Figure 5-20 50% Blocking InfiniBand
Filler panel: Filler panels (part number 00Y3011) are placed in rack unit 21
and 41 to prevent hot air recirculation.
148
IBM NeXtScale System Planning and Implementation Guide
48 Port 1GBe 36 Down 2 10G Up
36 Port IB 18 Down 18 UP
12
10
8
Chassis 6
6
4
2
12
10
Chassis 5 8
6
4
2
12
10
8
Chassis 4
6
4
2
36 Port IB 18 Down 18 UP
12
10
8
Chassis 3
6
4
2
12
10
8
Chassis 2
6
4
2
12
10
8
Chassis 1
6
4
2
36 Port IB 18 Down 18 UP
48 Port 1GBe 36 Down 2 10G Up
Building Block 2
48 Port 1GBe 36 Down 2 10G Up
Building Block 3
Cable Channel 4
Building Block 2
Cable Channel 2
Building Block 1
48 Port 1GBe 36 Down 2 10G Up
36 Port IB 18 Down 18 UP
Chassis 6
Chassis 5
Chassis 4
36 Port IB 18 Down 18 UP
Chassis 3
Chassis 2
Chassis 1
36 Port IB 18 Down 18 UP