IBM AIX HACMP SG24-5131-00 Hardware User Manual page 209

With reference to Figure 17 above, imagine two nodes, Node X and Node Y,
running the same application. The nodes are connected by the switch and
have locally-attached disks. On Node X's disk resides a volume group
containing the raw logical volume lv_X. Similarly, Node Y has lv_Y. For the
sake of illustration, let us suppose that lv_X and lv_Y together constitute an
Oracle Parallel Server database to which the application on each node makes
I/O requests.
The application on Node X requests a piece of data in the database. After the
node's Virtual Memory Manager (VMM) determines that the data is not in
memory, it talks not to the regular Logical Volume Manager (LVM), but rather
to the VSD device driver. The VSD device driver is loaded as a kernel
extension. Thus VSDs configured in the SP are known to the appropriate
nodes at the kernel level.
The VSD device driver can fetch the data from one of three places:
1. From the VSD cache, if the data is still there from previous requests. VSD
cache is shared by all VSDs configured on a node. Data is stored in 4KB
blocks, a size optimized for Oracle Parallel Server. If your I/O patterns
involve I/O operations larger than 4KB, we recommend disabling VSD
cache, because its management becomes counterproductive.
2. From lv_X, in which case the VSD device driver exploits Node X's normal
LVM and Disk Device Driver (Disk DD) pathway to fetch the data.
3. From lv_Y, in which case the VSD device driver issues the request through
the IP and Network Device Driver (Net DD) pathway to access Node Y. For
performance, VSD uses its own stripped-down IP protocol. Once the
request is passed up through Node Y's Net DD and IP layers, Node Y's
VSD device driver accesses the data either from VSD cache or from lv_Y.
The VSD server node uses the buddy buffer to temporarily store data for I/O
operations originating at a client node, and to handle requests that are
greater than the IP message size. In contrast to the data in the cache buffer,
the data in a buddy buffer is purged immediately after the I/O operation
completes. Buddy buffers are used only when a shortage in the switch buffer
pool occurs, or, on certain networks with small IP message sizes (for
example, Ethernet). The maximum and minimum size for the buddy buffer
must be defined when the VSD is created. For best performance, you must
ensure that your buddy buffer limits accommodate your I/O transaction sizes
to minimize the packetizing workload of the VSD protocol. Buddy buffers are
discussed in detail in IBM Parallel System Support Programs for AIX
Managing Shared Disks , SA22-7279.
Special RS/6000 SP Topics
191