IBM Power Systems 775 Manual page 46

Octant 0 Octant 1 Octant2
Octant 0 Octant 1 Octant2
Octant 0 Octant 1 Octant2
Octant 0 Octant 1 Octant2
.
Figure 1-19 Second level wiring connector count
1.4.14 Power 775 system
This section describes the Power 775 system and provides details about the third level of
interconnect.
Third level interconnect: Distance
This level connects every Supernode to every other Supernode in a system by using Hub
module optical links. Each Supernode includes up to 512 D-links, which allows for system that
contains up to 512 Supernodes. Every Supernode features a minimum of one hop D-link to
every other Supernode, and there are multiple two hop connections, as shown in Figure 1-20
on page 33.
Each HUB contains 16 Optical D-Links. The Physical node (board) contains eight HUBs;
therefore, a physical node (board) contains 16 x 8 = 128 Optical D-Links. A Super Node is
four Physical Nodes, which result in 16 x 8 x 4 = 512 Optical D-Links per Super node. This
configuration allows up to 2048 CEC connected drawers.
In smaller configurations, in which the system features less than 512 Super Nodes, more than
one optical D-Link per node is possible. Multiple connections between Supernodes are used
for redundancy and higher bandwidth solutions.
32
IBM Power Systems 775 for AIX and Linux HPC Solution
Octant3 Octant 4 Octant 5
Node 4
Octant3 Octant 4 Octant 5
Node 3
Octant3 Octant 4 Octant 5
Node 2
Octant3 Octant 4 Octant 5
Node 1
Octant 6 Octant 7
Octant 6 Octant 7
8 x 8 =64
Connections
Octant 6 Octant 7
8 x 16 =128
Connections
Octant 6 Octant 7
8 x 24 =192
Connections
Step 4
The total number of connections to build a super
node are 384. 192 + 128 + 64 = 384
It must be noted that every Octant has 24
connections, but the total number of connections
across the 4 nodes in a given Super Node is 384.
Step 3
Step 1 & 2 below have connected Node 1 & Node 2
to every other Octant in the Super Node. We need
now to connect Node 3 to Node 4. To do this every
Octant in Node 3 needs 8 connections to the 8
octants in Node 4 which results in 64 connections. At
this point every Octant in the Super Node is
connected to every other Octant in the Super Node
Step 2
Step One Below has connected Node 1 to every
other Octant in the Super Node. We need now to
connect Node 2 to every other remaining Node
(nodes 3 &4) in the Super Node.
This requires 16 connections from each of the 8
Octant in Node 2. So every Octant in Node 2 has 16
connections from it and there are 8 Octants resulting
in 128 connections
Step 1
Each Octant in Node 1 needs to be connected to the
8 Octants in Node 2, the 8 Octants in Node 3, and
the 8 Octants in Node 4. This requires 24 connection
from each of the 8 Octant in Node 1. So every
Octant has 24 connections from it and there are 8
Octants resulting in 192 connections