Chelsio Communications Terminator Series Installation And User Manual page 95

Chapter IV. iWARP (RDMA)
The performance is best with NIC MTU set to 9000 bytes.
• Run Open MPI application as:
mpirun --host node1,node2 -mca btl openib,sm,self /usr/mpi/gcc/openmpi-
x.y.z/tests/imb/IMB-MPI1
For OpenMPI/RDMA clusters with node counts greater than or equal to 8 nodes, and
Note
process counts greater than or equal to 64, you may experience the following RDMA
address resolution error when running MPI jobs with the default OpenMPI settings:
The RDMA CM returned an event error while attempting to make a connection.
This type of error usually indicates a network configuration error.
Local host: core96n3.asicdesigners.com
Local device: Unknown
Error name: RDMA_CM_EVENT_ADDR_ERROR
Peer: core96n8
Workaround: Increase the OpenMPI rdma route resolution timeout. The default is 1000, or
1000ms. Increase it to 30000 with this parameter:
--mca btl_openib_connect_rdmacm_resolve_timeout 30000
openmpi-1.4.3 can cause IMB benchmark stalls due to a shared memory BTL
Important
issue. This issue is fixed in openmpi-1.4.5 and later releases. Hence, it is
recommended that you download and install the latest stable release from
Open MPI's official website,
• Run MVAPICH2 application as :
mpirun_rsh -ssh -np 8 -hostfile mpd.hosts $MVAPICH2_HOME/tests/imb/IMB-MPI1
Chelsio Unified Wire for Linux
http://www.open-mpi.org
95