Table of Contents
Advertisement
When source and destination hubs lie in different supernodes, as shown in Figure 1-29, the
following conditions exist:
Route possibilities: one D-hop, or L-D (L-D-L routes also are used)
LNMC needs non-local link status to construct L-D routes
Figure 1-29 L-D route example
The ISR also supports indirect routes to provide increased bandwidth and to prevent hot
spots in the interconnect. An indirect route is a route that has an intermediate compute node
in the route that is on a different supernode, not the same supernode in which source and
compute nodes reside. An indirect route must employ the shortest path from the source
compute node to the intermediate node, and the shortest path from the intermediate compute
node to the destination compute node. Although the longest indirect route has five hops at
most, no more than three hops are L hops and two hops (at most) are D hops. This
configuration often is represented as an L-D-L-D-L route.
The following methods are used to select a route is when multiples routes exist:
Software specifies the intermediate supernode, but the hardware determines how to route
to and then route from the intermediate supernode.
The hardware selects among the multiple routes in a round-robin manner for both direct
and indirect routes.
The hub chip provides support for route randomization in which the hardware selects one
route between a source–destination pair. Hardware-directed randomized route selection is
available only for indirect routes.
These routing modes are specified on a per-packet basis.
The correct choice between the use of direct- versus indirect-route modes depends on the
communication pattern that us used by the applications. Direct routing is suitable for
communication patterns in which each node must communicate with many other nodes by
using spectral methods. Communication patterns that involve small numbers of compute
nodes benefit from the extra bandwidth that is offered by the multiple routes with indirect
routing.
Chapter 1. Understanding the IBM Power Systems 775 Cluster
39
Advertisement
Table of Contents
