IBM Power Systems 775 Manual page 136

So that the performance improvements are seen, the application must use at least four
striping instances (MP_EUIDEVICE must be set to sn_all and the value that is specified for
MP_INSTANCES must greater than four).
Controlling the number of immediate send buffers
By using immediate send rather than FIFO send, you significantly reduce the latency of a
send/receive message. You request the number of immediate send buffers to use for your
application by using the MP_IMM_SEND_BUFFERS environment variable or the
-imm_send_buffers command line flag. You specify any number that is greater than or equal
to zero, and less than or equal to the maximum number of available send buffers on the HFI
(the default value is 1).
There are 128 immediate send buffers per HFI, and these buffers are shared between
windows. If there is only a single LPAR on the octant, there are 128 available hardware
buffers. An octant with four LPARs has 32 available hardware buffers. The sum of the buffers
that are used by each window might not exceed the total number of buffers that are available
on the HFI.
Using RDMA
As shown in Figure 2-7, the Remote Direct Memory Access (RDMA) is a mechanism that
allows large contiguous messages to be transferred while the message transfer overhead is
reduced. PE support for RDMA differs, depending on the operating system you are using.
Figure 2-7 Remote Direct Memory Access model
RDMA is supported by the HFI (PE for AIX only).
Using RDMA with the Host Fabric Interface
To use RDMA with the Host Fabric Interface (HFI), you must perform the following tasks:
Verify that MP_DEVTYPE is set to hfi.
Request the use of bulk transfer by completing one of the following tasks:
– Set the MP_USE_BULK_XFER environment variable to yes:
MP_USE_BULK_XFER=yes
122
IBM Power Systems 775 for AIX and Linux HPC Solution