Memory Technologies - HP DL585 - ProLiant - G2 Technology Brief

Memory technologies

Typical multiprocessor PC server architecture connects IA-32 processors to memory DIMMs by means
of a north bridge chip. The north bridge is a memory controller and bridge to the I/O expansion
interface. AMD Opteron architecture differs from typical multiprocessor server architecture in that the
memory controller is integrated into the processor to boost performance. Performance is enhanced by
eliminating bus contention created when memory and I/O paths pass through a typical,
non-integrated north bridge. Because the memory controller is integrated onto the processor chip,
memory latency is greatly reduced.
In the DL585 G2, processors access the memory controllers at core speed. Each on-chip memory
controller directly accesses the DIMMs on the same processor/memory board at the DIMM speed.
The aggregate bandwidth for system-accessible memory increases with the number of processors.
The Opteron chipset supports dual-channel memory, which reduces memory latency by increasing the
bus width from 64 bits to 128 bits. Each memory controller has two 64-bit-wide memory channels,
and the channels operate in parallel to support the 128-bit interface. Because the DL585 G2 has
dual-width memory channels, DIMMs must be installed in pairs.
On-die memory controllers result in faster memory accesses by each processor. The memory controller
on each processor has direct access to the memory attached directly to that controller. However, for a
process thread running on one processor to access memory attached to another processor, extra
coordination is required.
To provide optimum performance for a wide variety of applications, the DL585 G2 can support either
of two methods of organizing memory access: linear, non-uniform memory access (NUMA) or node
interleaving, sufficiently uniform memory accessing (SUMA).
A node consists of one or more processors, its embedded memory controller, and the attached
DIMMs. The total memory attached to all the processors is divided into 4-KB segments. In the case of
linear addressing (NUMA), consecutive 4-KB segments are on the same node. In the case of node
interleaving (SUMA), consecutive 4-KB segments are on different or adjacent nodes.
Linear memory accessing (NUMA) defines the memory on all nodes sequentially. It assigns sequential
addresses to all memory locations on node 0, then to all the memory locations on node 1, and so on
until memory locations on all nodes have been assigned.
Node interleaving (SUMA) breaks memory into 4-KB addressable entities. Addressing starts with
address 0 on node 0. Sequential addresses through address 4095 are assigned to node 0, addresses
4096 through 8191 to node 1, addresses 8192 through 12287 to node 2, and addresses 12888
through 16383 to node 3. Address 16384 is assigned to node 0, and the process continues until all
memory has been assigned in this fashion.
To take advantage of NUMA architecture, the operating system and the applications that run on the
system must assign memory on a per-thread basis. In general, a NUMA-aware operating system such
as Microsoft Windows and a NUMA-aware application such as Microsoft SQL Server will benefit
from the NUMA organization. A NUMA-aware operating system and applications that allocate and
de-allocate memory at the thread level will benefit from the NUMA organization because the
allocation and the thread will have a tendency to run on the same node. If an application uses a
common allocation thread, it will benefit from node interleaving.
The DL585 G2 has, by default, NUMA memory configuration. For those applications that cannot take
advantage of the NUMA architecture, performance may be improved by activating node interleaving
(SUMA). System administrators can activate node interleaving using the HP ROM-Based Setup Utility
(RBSU) that is provided as part of the HP ProLiant Essentials Foundation Pack.
5