System Infiniband Switch Blades; Enclosure Switch Density Choices - Silicon Graphics ICE X User Manual

3: System Overview

System InfiniBand Switch Blades

Enclosure Switch Density Choices

36
• The number of DIMMs per channel.
• The DIMM speed - the DIMM itself has a maximum operating frequency or speed, such as
2133 MT/s or 1600 MT/s.
Note: A DIMM must be rated for the maximum speed to be able to run at the maximum
speed. For example: a single 1600 MT/s DIMM on a channel will only operate at speeds up
to 1600 MT/s - not 2133 MT/s.
Populating one 1600 MT/s DIMM on each channel of an ICE X system blade node board delivers
a maximum of 12.8 GB/s per channel or 51.2 GB/s total memory bandwidth.
A minimum of one dual-inline-memory module (DIMM) is required for each processor on a node
board; four DIMMs per processor are recommended. Each of the DIMMs on a blade's node board
must be the same capacity and functional speed. When possible, it is generally recommended that
all blade node boards within an enclosure use the same number and capacity (size) DIMMs.
Each blade in the enclosure pair may have a different total DIMM capacity. For example, one
blade may have 16 DIMMs, and another may have only eight. Note that while this difference in
capacity is acceptable functionally, it may have an impact on compute "load balancing" within the
system.
Two or four fourteen-data-rate (FDR) InfiniBand switch blades can be used with each blade
enclosure pair configured in the SGI ICE X system. Single-node blade enclosure pairs use two
switch blades for single-plane InfiniBand topologies. Enclosure pairs with four switch blades
(using single-node blades) use a dual-plane topology that provides high-bandwidth
communication between compute blades inside the enclosure as well as blades in other enclosures.
Enclosure pairs using dual-node compute blades (such as the IP115 or IP133) must use four switch
blades to support a single-plane InfiniBand topology.
Each SGI ICE X system comes with a choice of two switch blade configurations.
• Single 36-port FDR IB ASIC (standard) with 18 external ports per switch
• Dual 36-port FDR IB ASIC (premium) with a total of 48 external ports per switch
007-5806-004