Upgrading Ib Software - Qlogic Fast Fabric User Manual

Q
At this point the user is ready to move onto full scale HPL runs. Assorted sample
HPL.dat files are provided in /opt/iba/src/mpi_apps/hpl-config. These files are a
good starting point for most clusters and should get within 10-20% of the optimal
performance for the cluster. The problem sizes used assume a cluster with 1GB of
physical memory per processor (e.g., for a 2 processor node, 2GB of node memory
is assumed). For each cluster size, 4 files are provided:
t - a very small test run (5000 problem size)
s - a small problem size on the low end of optimal problem sizes
m - a medium problem size
l - a large problem size
These can be selected using config_hpl. The following command displays the
preconfigured problem sizes available:
./config_hpl
For example, to do a small run for a 256 processor cluster (i.e., 128 nodes of dual
CPU systems):
1. Type ./config_hpl 256s and press Enter.
2. Type ./run_hpl 256 and press Enter.
During these runs the user should use top on a node to monitor memory and CPU
usage. The xhpl should use 98-99% of the CPU. If any other processes are taking
more than 1-2%, review the host configuration and stop these extra processes if
possible. HPL is very sensitive to swapping. If a lot of swapping is seen, and xhpl
is dropping below 97% for long durations, this may indicate a problem size that is
too large for the memory and OS configuration.
At this point the user can continue to tune HPL to refine performance. Parameters
in /opt/iba/src/mpi_apps/mpi.param.hpl and in HPL.dat can all affect HPL
performance. In addition the selection of compiler and BLAS Math library may also
significantly affect performance. The new HPL.dat files may be placed in
/opt/iba/src/mpi_apps/hpl-config and use config_hpl to select them and copy them
to all nodes in the run. Alternately scpall may be used to copy the file to all nodes.
Refer to the section "Basic Setup and Administration Tools" on page 5-11 for more
information on scpall.
3.12

Upgrading IB software

If an existing cluster which has been installed and verified needs to have Infiniband
software upgraded, the following steps may be followed.
1. (All) On each IB Management Node, perform an upgrade installation of the
Fabric Access Software using the procedure documented in the Fabric Access
Software Users Guide. Each IB Management Node must have at least Fast
Fabric, the IB Stack and IPoIB installed and configured. For MPI clusters the
D000006-000 Rev A
3 – Getting Started
3-21