Page of 15

HP 4X - DDR InfiniBand Mezzanine HCA Using Manual

Using infiniband for a scalable compute infrastructure.
Hide thumbs
Using InfiniBand for a
scalable compute infrastructure
technology brief, 3
rd
Abstract .............................................................................................................................................. 2
Introduction ......................................................................................................................................... 2
InfiniBand technology ........................................................................................................................... 4
InfiniBand components ...................................................................................................................... 5
InfiniBand software architecture ......................................................................................................... 5
InfiniBand hardware architecture ....................................................................................................... 8
Link operation .................................................................................................................................. 9
Scale-out clusters built on InfiniBand and HP technology ......................................................................... 11
Conclusion ........................................................................................................................................ 13
Appendix A: Glossary ........................................................................................................................ 14
For more information .......................................................................................................................... 15
Call to action .................................................................................................................................... 15
edition
MPI ............................................................................................................................................. 7
IPoIB ........................................................................................................................................... 7
RDMA-based protocols .................................................................................................................. 7
RDS ............................................................................................................................................ 8

Advertising

   Related Manuals for HP 4X - DDR InfiniBand Mezzanine HCA

   Summary of Contents for HP 4X - DDR InfiniBand Mezzanine HCA

  • Page 1

    RDMA-based protocols ........................7 RDS ............................8 InfiniBand hardware architecture ....................... 8 Link operation ..........................9 Scale-out clusters built on InfiniBand and HP technology ................. 11 Conclusion ............................13 Appendix A: Glossary ........................14 For more information .......................... 15 Call to action ............................ 15...

  • Page 2: Abstract, Introduction

    (HPC) but also for financial services and Oracle-based database applications. InfiniBand (IB) is one of the most important technologies that enable the adoption of cluster computing. This technology brief describes InfiniBand as an interconnect technology used in cluster computing, provides basic technical information, and explains the advantages of implementing InfiniBand-based scale-out architectures.

  • Page 3

    Scale-out systems allow infrastructure architects to meet performance and cost goals, but interconnect performance, scalability, and reliability are key areas that must be carefully considered. A cluster infrastructure works best when built with an interconnect technology that scales easily, reliably, and economically with system expansion.

  • Page 4: Infiniband Technology

    Packets can contain data payloads of up to four kilobytes. The packets are passed to the network layer, which selects a route to the destination node and, if necessary, attaches the route information to the packets. The data link layer attaches a local identifier (LID) to the packet for communication at the subnet level.

  • Page 5: Infiniband Components, Infiniband Software Architecture

    The layered software architecture of the HCA allows writing code without specific hardware in mind. The functionality of an HCA is defined by its verb set, which is a table of commands used by the application programming interface (API) of the operating system being run. A number of services and software protocols are available (Figure 4) and, depending on type, can be implemented from user space or from the kernel.

  • Page 6

    Open Fabrics Verbs and API Provider Hardware-Specific Driver Hardware InfiniBand HCA As indicated in Figure 4, InfiniBand supports a variety of upper level protocols (ULPs) and libraries that have evolved since the introduction of InfiniBand. The key protocols are discussed in the following pages.

  • Page 7: Ipoib, Rdma-based Protocols

    InfiniBand. The iSER protocol provides iSCSI manageability to RDMA storage operations. NFS – The Network File System (NFS) is a storage protocol that has evolved since its inception in the 1980s, undergoing several generations of development while remaining network-independent. With...

  • Page 8: Infiniband Hardware Architecture

    The bi-directional links contain dedicated send and receive lanes for full duplex operation. For quad data rate (QDR) operation, each lane has a signaling rate of 10 Gbps. Bandwidth is increased by adding more lanes per link. InfiniBand interconnect types include 1x, 4x, or 12x wide full-duplex links (Figure 5)., The 4x is the most popular configuration and provides a theoretical full-...

  • Page 9: Link Operation

    4X DDR products is widespread, and deployment of QDR systems is expected to increase. Link operation Each link can be divided (multiplexed) into a set of virtual lanes, similar to highway lanes (Figure 7). Each virtual lane provides flow control and allows a pair of devices to communicate autonomously.

  • Page 10

    When a connection between two channel adapters is established, one of the following transport layer communication protocols is selected: Reliable connection (RC) – data transfer between two entities using receive acknowledgment Unreliable connection (UC) – same as RC but without acknowledgement (rarely used) Reliable datagram (RD) –...

  • Page 11: Scale-out Clusters Built On Infiniband And Hp Technology

    The only operation available for the receive queue is Post Receive Buffer transfer, which identifies a buffer that a client may send to or receive from using a Send or RDMA Write data transfer. Scale-out clusters built on InfiniBand and HP technology In the past few years, scale-out cluster computing has become a mainstream architecture for high performance computing.

  • Page 12

    HP Oracle Database Machine HP BladeSystem for Oracle Optimized Warehouse (OOW) HP Cluster Platforms are built around specific hardware and software platforms and offer a choice of interconnects. For example, the HP Cluster Platform CL3000BL uses the HP BL2x220c G5, BL280c G6, and BL460c blade servers as the compute node with a choice of GbE or InfiniBand interconnects.

  • Page 13: Conclusion

    While zero-copy (RDMA) protocols have been applied to TCP/IP networks such as Ethernet, RDMA is a core capability of InfiniBand architecture. Flow control support is native to the HCA design, and the latency time for InfiniBand data transfers is generally less—approaching 1 microsecond for MPI pingpong latency—than that for...

  • Page 14: Appendix A: Glossary

    Description Application Programming Interface: software routine or object written in support of a language Double Data Rate: for InfiniBand, clock rate of 5.0 Gbps (2.5 Gbps x 2) Gigabit Ethernet: Ethernet network operating at 1 Gbps or greater Host Channel Adapter: hardware interface connecting a server node to the IB network...

  • Page 15: For More Information, Call To Action

    TechCom@HP.com © 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty.

This manual also for:

489183-b21 - infiniband ddr switch, 4x

Comments to this Manuals

Symbols: 0
Latest comments: