Architectural Elements - HP zl User Manual

devices. This protocol allows InfiniBand hosts to natively send SCSI commands as if the storage was
direct attached.
The SRP protocol is designed to operate using an RDMA communication service. An RDMA
communication service provides communication between pairs of consumers; it uses messages for
control information and RDMA operations for data transfers.
The SRP protocol is only used if you have a Fibre Channel Gateway installed in your InfiniBand
system.
uDAPL
The user Direct Access Programming Library (uDAPL) is a standardized user mode API that natively
supports InfiniBand fabrics.
uDAPL performs name to address translations, establishes connections, and transfers data reliably.
The primary responsibilities of uDAPL are:
• Connection management
• Low latency data transfer and completion
MPI
The MPI protocol is bundled with the Upper Layer Protocol (ULP) suite. Topspin has taken the Ohio
State University's (OSU's) MVAPICH and created Topspin's version of this release. However, in
addition, the HCAs also run using other popular InfiniBand MPI implementations.
Alternative MPI Implementations
Topspin customers have also deployed a variety of MPIs that use Mellanox's VAPI layer. This includes
OSU, LAM-MPI, Verari Systems Software, Inc's MPI/Pro (formerly Softech's ), and LANL MPI.
Topspin products have also been used successfully with SCALI MPI, which is based on uDAPL.
Differences Between Topspin and Standard MPI
There are significant differences between the version of MPI provided, and OSU's MPI.
• There is no restriction on which HCA port is used (OSU only supports Port 1)
• Support for Opteron 64 bit operation is provided
• Bug fixes have been provided for the purpose of improving stability

Architectural Elements

What is RDMA?
InfiniBand utilizes Remote Direct Memory Access (RDMA) technology. RDMA is a technology that
allows one computer to place information directly into the memory of another computer.
RDMA is specifically characterized by two important features:
• allows user space applications to directly access hardware
• zero-copy data movement
A combination of hardware and software allows user space applications to read and write the memory
of a remote system without kernel intervention or unnecessary data copies. This results in lower CPU
utilization per I/O operation and more efficient use of machine resources because applications place
most of the messaging burden upon InfiniBand's high-speed network hardware.
Work Queues and Queue Pairs
A "verb" is the abstract description that is used to define the functionality of the Host Channel Adapter
(HCA). A "verb consumer" refers to the direct user of the verb.
A work queue provides a verb consumer with the ability to queue up a set of instructions that are
executed by the Channel Adapter. There are two types of Work Queues: Send Work Queue (outbound)
and a Receive Work Queue (inbound). Together these Work Queues create a Queue Pair.
3