Identifying Infiniband Fabric Components - HP Cluster Platform Interconnects v2010 User Manual

— Security and partitioning
— Integrated management tools
• A strong technical future, with support for:
— PCI cards:
◦
◦
◦
◦
◦
◦
— Remote direct memory access (RDMA)
— Faster link speeds
InfiniBand Protocol
InfiniBand fabrics provide a single point of management for all HCAs (host channel adapters)
in the fabric. Management utilities enable you discover and configure devices, handle device or
path failures and allocate resources. InfiniBand enables applications to access the HCA directly
without involving the operating system by mapping the application memory to the physical
memory in tables that are maintained in the HCA.
An applications programming interface (API) facilitates control of HCAs and enables reliable
message passing between the channel adapters in different devices. The entire transport layer
is implemented in hardware (on integrated circuits, rather than is a software layer), Each channel
adapter provides queue pairs, for read and write that are accessible directly by programs. Each
queue pair provides a reliable connection between connected applications, nodes, or an unreliable
datagram message queue. The communications protocol consists of four operations:
• Transmit a message.
• Using remote data memory access (RDMA), writes data directly into a device's memory.
• Using remote data memory access (RDMA), reads data directly from a device's memory.
• Atomic operations, updating remote memory words.
The HCA device provides the following functions:
• Receives and posts the following operations to a queue as work queue elements:
— Messages.
— RDMA operations.
• Handles notifications and completion events through completion queues. To reduce the
volume of events in the system, completion queues might be shared for several queue pairs.
The remaining sections in this chapter include the following topics:
•

Identifying InfiniBand fabric components

• ISR 9xxxx-series Interconnects
• Identifying the ISR 9024 interconnect
• Identifying the ISR 9024 S/D RoHS compliant interconnect
• Identifying the ISR 9096 interconnect
• Identifying the ISR 9288 interconnect
• Understanding the interconnect management capabilities
1.1 Identifying InfiniBand Fabric Components
Figure 1-1 shows a simplified model of an InfiniBand fabric and identifies the following typical
fabric components:
• HCA (Host Channel Adapter), which can have two hardware formats:
20 InfiniBand Technology Overview
Voltaire SDR PCI-X
Topspin/Mellanox SDR PCI-X Rev B; Mellanox SDR PCI-X Rev C
Topspin/Mellanox SDR PCI-Express Rev B; Mellanox SDR PCI-Express Rev C
Mellanox Mem Free SDR PCI-Express
Mellanox Mem Free DDR PCI-Express
Mellanox DDR PCI-Express
(Section 1.1).
(Section 1.2).
(Section 1.3).
(Section 1.5).
(Section 1.6).
(Section 1.4).
(Section 1.7).