Resource Management In Numa Architectures - VMware vSphere ESXi 4.0 Management Manual

vSphere Resource Management Guide
When a virtual machine moves to a new node, the ESX/ESXi host immediately begins to migrate its memory
in this fashion. It manages the rate to avoid overtaxing the system, particularly when the virtual machine has
little remote memory remaining or when the destination node has little free memory available. The memory
migration algorithm also ensures that the ESX/ESXi host does not move memory needlessly if a virtual machine
is moved to a new node for only a short period.
When initial placement, dynamic rebalancing, and intelligent memory migration work in conjunction, they
ensure good memory performance on NUMA systems, even in the presence of changing workloads. When a
major workload change occurs, for instance when new virtual machines are started, the system takes time to
readjust, migrating virtual machines and memory to new locations. After a short period, typically seconds or
minutes, the system completes its readjustments and reaches a steady state.
Transparent Page Sharing Optimized for NUMA
Many ESX/ESXi workloads present opportunities for sharing memory across virtual machines.
For example, several virtual machines might be running instances of the same guest operating system, have
the same applications or components loaded, or contain common data. In such cases, ESX/ESXi systems use a
proprietary transparent page-sharing technique to securely eliminate redundant copies of memory pages. With
memory sharing, a workload running in virtual machines often consumes less memory than it would when
running on physical machines. As a result, higher levels of overcommitment can be supported efficiently.
Transparent page sharing for ESX/ESXi systems has also been optimized for use on NUMA systems. On NUMA
systems, pages are shared per-node, so each NUMA node has its own local copy of heavily shared pages. When
virtual machines use shared pages, they don't need to access remote memory.
Memory Page Sharing Across and Within NUMA Nodes
The VMkernel.Boot.sharePerNode option controls whether memory pages can be shared (de-duplicated) only
within a single NUMA node or across multiple NUMA nodes.
VMkernel.Boot.sharePerNode is turned on by default, and identical pages are shared only within the same
NUMA node. This improves memory locality, because all accesses to shared pages use local memory.
N This default behavior is the same in all previous versions of ESX.
OTE
When you turn off the VMkernel.Boot.sharePerNode option, identical pages can be shared across different
NUMA nodes. This increases the amount of sharing and de-duplication, which reduces overall memory
consumption at the expense of memory locality. In memory-constrained environments, such as VMware View
deployments, many similar virtual machines present an opportunity for de-duplication, and page sharing
across NUMA nodes could be very beneficial.

Resource Management in NUMA Architectures

You can perform resource management with different types of NUMA architecture. The systems that offer a
NUMA platform to support industry-standard operating systems include those based on either AMD CPUs
or the IBM Enterprise X-Architecture.
IBM Enterprise X-Architecture
One architecture that supports NUMA is the IBM Enterprise X-Architecture.
76 VMware, Inc.