Nvidia ConnectX-5 User Manual
  1. Manuals
  2. Brands
  3. Nvidia Manuals
  4. Adapter
  5. Mellanox ConnectX-5
  6. User manual
Nvidia ConnectX-5 User Manual

Nvidia ConnectX-5 User Manual

Infiniband/ethernet socket direct adapter cards
Hide thumbs Also See for ConnectX-5:
Table of Contents

Advertisement

Quick Links

Download this manual
NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct
Adapter Cards User Manual

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the ConnectX-5 and is the answer not in the manual?

Questions and answers

Related Manuals for Nvidia ConnectX-5

Summary of Contents for Nvidia ConnectX-5

  • Page 1 NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...

  • Page 2: Table Of Contents

    Troubleshooting Updating Adapter Firmware Updating Adapter Firmware Specifications Specifications Monitoring Monitoring Finding the GUID/MAC and Serial Number on the Adapter Card Finding the GUID/MAC and Serial Number on the Adapter Card NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 3 Figure 15. Image2019 1 10 17 20 6 Figure 16. Image2019 1 10 17 20 53 Figure 17. Image2018 11 14 15 55 33 Figure 18. Image2019 1 10 17 21 39 NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 4 Figure 24. Image2019 1 10 18 22 55 Figure 25. Image2019 1 13 10 58 45 Figure 26. Image2019 1 10 18 27 38 Figure 27. Image2019 1 10 18 28 17 NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 5 Table 2. Table 3. Table 4. Table 5. Table 6. Table 7. Table 8. Table 9. Table 10. Table 11. Table 12. Table 13. Table 14. Table 15. Table 16. Table 17. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 6 User Manual provides details as to the interfaces of the adapter card, specifications, required software and firmware for operating the adapter card, and relevant documentation EOL'ed (End of Life) Ordering Part Numbers The table below provides the ordering part numbers (OPN) for the available ConnectX-5 InfiniBand/Ethernet Socket Direct adapter cards. NVIDIA Legacy...
  • Page 7 This manual is intended for the installer and user of these cards. The manual assumes basic familiarity with InfiniBand and Ethernet network and architecture specifications. Technical Support Customers who purchased NVIDIA products directly from NVIDIA are invited to contact us through the following methods: URL: https://www.nvidia.com > Support E-mail: Network-support@nvidia.com...
  • Page 8 NVIDIA ConnectX® NATIVE ESXi stack. See VMware® ESXi Drivers Ethernet Documentation. User Manual NVIDIA Firmwar e Utility (mlxup) NVIDIA firmware update and query utility used to update the firmware. Refer User to Firmware Utility (mlxup) Documentation. Manual Release Notes NVIDIA Firmwar e Tools User Manual describing the set of MFT firmware management tools for a...
  • Page 9 Solution splitter cables, Active Optical Cables (AOCs) and transceivers in a wide range of lengths from 0.5m to 10km. In addition to meeting IBTA standards, NVIDIA tests every product in an end-to-end environment ensuring a Bit Error Rate of less than 1E-15.Read more at LinkX Cables and Transceivers.

  • Page 10: Introduction

    Device number. Each of the PCIe x8 busses sees two network ports; in effect, the two 100Gb/s physical ports of the ConnectX-5 Socket Direct Adapter are viewed as four netdevices by the system, with each netdevice providing a maximum of 50Gb/s data transfer rate.
  • Page 11 Full performance of the adapter card (100Gb/s) is possible only when installing both the adapter card and the auxiliary PCIe connection card in the dual PCIe x8 connectors.Please refer to the performance optimization document for further guidance: https://community.nvidia.com/s/article/understanding-pcie- configuration-for-maximum-performance NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 12 Uses PCIe Gen 3.0 (8GT/s) through two x8 edge connectors. Gen 1.1 and 2.0 Express compatible. (PCIe) 100Gb/s ConnectX-5 offers the highest throughput Ethernet adapter, supporting EDR InfiniBa 100Gb/s InfiniBand and 100Gb/s Ethernet and enabling any standard nd/Ethe networking, clustering, or storage to operate seamlessly over any converged rnet network leveraging a consolidated software stack.
  • Page 13 Network solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-5 effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.
  • Page 14 NVMe over Fabric offloads for target machine Erasure Coding T10-DIF Signature Handover ConnectX-5 SR-IOV technology provides dedicated adapter resources and SR-IOV guaranteed isolation and protection for virtual machines (VM) within the server. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 15 Direct PCIe stand-up adapter can be connected to a BMC using MCTP over SMBus or MCTP over PCIe protocols as if it is a standard NVIDIA PCIe stand-up adapter. For configuring the adapter for the specific manageability solution in use by the server, please contact NVIDIA Support.

  • Page 16: Interfaces

    Support for MSI/MSI-X mechanisms LED Interface There is one bi-color (yellow/green) I/O LED per port to indicate link status located on the adapter card. Physical and Logical Link Indications - Ethernet Mode: NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 17 Error Failure of I C access to the port Over-current conditions of the port Link Activity Blinks green reflecting the link speed Link Up Constant green light NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...

  • Page 18: Hardware Installation

    Hardware Installation Installation and initialization of ConnectX-5 adapter cards require attention to the mechanical attributes, power specifications, and precautions for electronic equipment. Safety Warnings Note Safety warnings are provided here in the English language. For safety warnings in other languages, refer to the Adapter Installation Safety Instructions.
  • Page 19 During periods of lightning activity, do not work on the equipment or connect or dis - connect cables. Copper Cable Connecting/Disconnecting Some copper cables are heavy and not flexible, as such, they should be carefully NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 20 This equipment should be installed, replaced, or serviced only by trained and qualified personnel. Equipment Disposal The disposal of this equipment should be in accordance to all national laws and regula - tions. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 21 IEC 60 825-1:1993 + A1:1997 + A2:2001 and EN 60825-1:1994+A1:1996+ A2:20 Installation Procedure Overview The installation procedure of ConnectX-5 Socket Direct adapter cards involve the following steps: Procedure Direct Link Check the system’s hardware and software...
  • Page 22 System Requirements Hardware Requirements Important Unless otherwise specified, NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous and dust (particulate) contamination. The operation environment should meet severity level G1 as per ISA 71.04 for gaseous contamination and ISO 14644-1 class 8 for...
  • Page 23 Airflow Requirements ConnectX-5 adapter cards are offered with two airflow patterns: from the heatsink to the network ports, and vice versa, as shown below. Please refer to Airflow Specifications for airflow numbers for each specific card model.
  • Page 24 Airflow from the network ports to the heatsink: Important All cards in the system should be planned with the same airflow direction. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 25 Software Requirements See Operating Systems/Distributions section under the Introduction section. Software Stacks - NVIDIA OpenFabric software package MLNX_OFED for Linux, WinOF-2 for Windows, and VMware. See the Driver Installation section. Unpacking the Product Important The adapter is being installed in a system that operates with voltages that can be lethal.
  • Page 26 Important Use the Slim-Line SAS harness and Auxiliary PCIe Connection card that are included in the ConnectX-5 Socket Direct adapter card package contents.For MCX556M-ECAT-S25, use the 25cm Slim-line SAS cable and its Auxiliary PCIe connection card.
  • Page 27 3. Save the two screws. Installing the New Bracket 1. Place the bracket onto the card until the screw holes line up. Important Do not force the bracket onto the adapter card. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 28 Please follow proper electrical grounding procedures. 2. Open the system case. 3. Connect the slim-line SAS connector (male) on the slim lineSAScable to the female connector on the auxiliary PCIe connection card. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 29 Important Use the Slim-Line SAS harness and auxiliary PCIe connection card that are supplied in the ConnectX-5 Socket Direct adapter card package contents. 4. Locate two available PCI Express slots on the server, one for the adapter card and one for the auxiliary PCIe connection card.
  • Page 30 PCIe connection card component-side facing the adapter card’s print-side. 5. Applying even pressure at both corners of the card, insert the auxiliary PCIe connection card into the PCI Express slot until firmly seated. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 31 Do not use excessive force when seating the card, as this may damage the system or the auxiliary PCIe connection card. 6. Secure the auxiliary PCIe connection card using the server's retention mechanisms. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 32 PCIe connection card. 8. Secure the adapter card using the server’s retention mechanisms. 9. Connect the slim line-SAS cable to the connector on the adapter card. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 33 10. Close the system case. Note NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 34 Warning When installing cables make sure that the latches engage. Important Always install and remove cables by pushing or pulling the cable and connector in a straight line with the card. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 35 1. Open Device Manager on the server. Click Start => Run, and then enter devmgmt.msc. 2. Expand System Devices and locate your NVIDIA ConnectX-5 adapter card. 3. Right click the mouse on your adapter's row and select Properties to display the adapter card properties window.
  • Page 36 5. In the Value display box, check the fields VEN and DEV (fields are separated by ‘&’). In the display example above, notice the sub-string “PCI\VEN_15B3&DEV_1003”: VEN is equal to 0x15B3 – this is the Vendor ID of NVIDIA; and DEV is equal to 1018 (for ConnectX-5) – this is a valid NVIDIA PCI Device ID.
  • Page 37 Please note that the following images are for illustration purposes only. Note Please note that the following images are for illustration purposes only. 1. Verify that the system is powered off and unplugged. 2. Wait 30 seconds. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 38 3. Disconnect the slim-line SAS cable from the connector on the adapter card. 4. Disengage the retention mechanisms (clips or screws) on the adapter card. 5. Holding the card from its center, gently pull the adapter card from the PCI Express slot. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 39 6. Disengage the retention mechanisms (clips or screws) on the auxiliary PCIe connection card. 7. Holding the card from its center, gently pull the auxiliary PCIe connection card from the PCI Express slot.d. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 40 8. Disconnect the slim-line SAS cable from the connector on the auxiliary PCIe connection card. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 41 NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...

  • Page 42: Driver Installation

    Driver Installation Please use the relevant driver installation section. Linux Driver Installation This section describes how to install and test the NVIDIA OFED for Linux package on a single host machine with Mellanox ConnectX-5 adapter hardware installed. Prerequisites Requirements Description Platforms A server platform with a ConnectX-5 adapter card installed.
  • Page 43 Since the two PCIe cards are installed in two PCIe slots, each card gets a unique PCI Bus and Device number. Each of the PCIe x8 busses sees two network ports; in effect, the two 100Gb/s physical ports of the ConnectX-5 Socket Direct Adapter are viewed as four netdevices by the system.
  • Page 44 flag during installation. --skip-unsupported-devices-check Usage ./mnt/mlnxofedinstall [OPTIONS] The installation script removes all previously installed OFED packages and re-installs from scratch. You will be prompted to acknowledge the deletion of the old packages. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 45 filesystem mounted over a ConnectX card, not regenerating the initramfs may even cause the system to fail to reboot. In such cases, the installer will recommend running the following command to update the initramfs: NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 46 /tmp directory. All Mellanox, OEM, OFED, or Distribution IB packages will be removed. Do you want to ?[y/N]:y continue See log file /tmp/mlnx_ofed_iso. .log 21642   Building OFED RPMs. Please wait... NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 47 For the list of installation options, run: ./mlnxofedinstall --h Installation Procedure This section describes the installation procedure of MLNX_OFED on NVIDIA adapter cards. 1. Log in to the installation machine as root. 2. Mount the ISO image on your machine.
  • Page 48 --forceFor example:./mlnxofedinstall -- without-dkms --add-kernel-support --kernel 3.13.0-85-generic -- without-fw-update --forceNote that the path to kernel sources (- -kernel-sources) should be added if the sources are not in their default location. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 49 firmware update can take effect. Refer to the table below to find the appropriate action for your specific card. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 50 files. See the relevant chapters in this manual for details. The list of the modules that will be loaded automatically upon boot can be found in the /etc/infiniband/openib.conf file. Warning NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 51 In case your machine has an unsupported network adapter device, no firmware update will occur and the error message below will be printed. "The firmware for this device is not distributed inside NVIDIA driver: 0000:01:00.0 (PSID: IBM2150110033) To obtain firmware for this device, please contact your HW vendor."...
  • Page 52 .IBMM2150110033.logs Driver Load Upon System Boot Upon system boot, the NVIDIA drivers will be loaded automatically. To prevent the automatic load of the NVIDIA drivers upon system boot: 1. Add the following lines to the "/etc/modprobe.d/mlnx.conf" file. blacklist mlx5_core blacklist mlx5_ib 2.
  • Page 53 OFED ISO image Note: If an adapter’s Flash was originally programmed with an Expansion ROM image, the automatic firmware update will also burn an Expansion ROM image. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 54 In case your machine has an unsupported network adapter device, no firmware update will occur and the error message below will be printed. "The firmware for this device is not distributed inside NVIDIA driver: 0000:01:00.0 (PSID: IBM2150110033) To obtain firmware for this device, please contact your HW vendor."...
  • Page 55 ///repos/MLNX_OFED/<MLNX_OFED file>/RPM-GPG-KEY-Mellanox Importing GPG key 0x6224C050 Userid: "Mellanox Technologies (Mellanox Technologies - Signing Key v2) <support@mellanox.com>" From : /repos/MLNX_OFED/<MLNX_OFED file>/RPM-GPG-KEY-Mellanox ok [y/N]: this 5. Check that the key was successfully imported. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 56 1. Log into the installation machine as root. 2. Mount the ISO image on your machine and copy its content to a shared location in your network. # mount -o ro,loop MLNX_OFED_LINUX-<ver>-<OS label>-<CPU arch>.iso /mnt NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 57 5. Create a YUM repository configuration file called "/etc/yum.repos.d/mlnx_ofed.repo" with the following content: [mlnx_ofed] name=MLNX_OFED Repository baseurl=file: ///<path to extracted MLNX_OFED package>/RPMS enabled= gpgcheck= 6. Check that the repository was successfully added. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 58 (with KMP package support) mlnx-ofed-bluefield-user-only.noarch : MLNX_OFED bluefield-user-only installer (User Space packages only) package mlnx-ofed-dpdk.noarch : MLNX_OFED dpdk installer (with KMP package support) mlnx-ofed-dpdk-upstream-libs.noarch : MLNX_OFED dpdk-upstream-libs installer (with KMP support) package NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 59 (with KMP package support) mlnx-ofed-vma-vpi-user-only.noarch : MLNX_OFED vma-vpi-user-only installer (User Space packages only package where: mlnx-ofed-all Installs all available packages in MLNX_OFED mlnx-ofed-basic Installs basic packages required for running NVIDIA cards NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 60 3.17 mlnx-ofed-vma- .fc21.x86_64.noarch : MLNX_OFED vma installer 3.17   kernel .fc21.x86_64 (without KMP support) package 3.17 mlnx-ofed-vma-eth- .fc21.x86_64.noarch : MLNX_OFED vma-eth 3.17 installer   kernel .fc21.x86_64 (without KMP support) package 3.17 NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 61 --> Running transaction check ---> Package mlnx-ofed-all.noarch will be installed --> Processing Dependency: kmod-isert = -OFED. .g832a737.rhel7u1   : mlnx-ofed-all- .noarch package ........qperf.x86_64   rds-devel.x86_64 1.12 rds-tools.x86_64 1.12 sdpnetstat.x86_64 1.60 srptools.x86_64   NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 62 file:/<path to extracted MLNX_OFED >/DEBS ./ package 4. Download and install NVIDIA's Technologies GPG-KEY. # wget -qO - http: //www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox | sudo apt- key add - 5. Verify that the key was successfully imported. NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 63   as a repository, then please this package note WARNING: that it contains unsigned rpms, therefore, you need to disable the gpgcheck WARNING: by setting in the repository conf file. 'gpgcheck=0' NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 64 - Userspace tools to restart and tune mlnx-ofed kernel modules mlnx-ofed-vma-vpi - MLNX_OFED vma-vpi installer (with DKMS support) package mlnx-ofed-kernel-only - MLNX_OFED kernel-only installer (with DKMS package support) mlnx-ofed-bluefield - MLNX_OFED bluefield installer (with DKMS package support) NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 65 Space packages only) mlnx-ofed-basic-exact - MLNX_OFED basic installer (with DKMS package support) (exact) mlnx-ofed-basic - MLNX_OFED basic installer (with DKMS support) package mlnx-ofed-dpdk-upstream-libs - MLNX_OFED dpdk-upstream-libs installer (with DKMS support) package where: NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 66 MLNX_OFED dpdk-upstream-libs installer package 2. Install the desired group. apt-get install '<group name>' Example: apt-get install mlnx-ofed-all Warning Installing MLNX_OFED using the “apt-get” tool does not automatically update the firmware.To update the firmware to NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 67 Depending on the application of the user's system, it may be necessary to modify the default configuration of network adapters based on the ConnectX® adapters. In case that tuning is required, please refer to the Performance Tuning Guide for NVIDIA Network Adapters.

  • Page 68: Troubleshooting

    Link light is on, but with Check that the latest driver is loaded Check that both the adapter and its link are set to the same speed communicati and duplex settings established NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 69 Download and install MFT: MFT Documentation Refer to the User Manual for installation instructions. Mellanox Firmware Once installed, run: Tool (MFT) mst start mst status flint -d <mst_device> q ibstat Ports Information ibv_devinfo NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 70 Firmware Version To download the latest firmware version, refer to the NVIDIA Upgrade Update and Query Utility. cat /var/log/messages dmesg >> system.log Collect Log File journalctl (Applicable on new operating systems) cat /var/log/syslog Windows Troubleshooting From the Windows desktop choose the Start menu and run:...

  • Page 71: Updating Adapter Firmware

    fixes. To check that your card is programmed with the latest available firmware version, download the mlxup firmware update and query utility. The utility can query for available NVIDIA adapters and indicate which adapters require a firmware update. If the user confirms, mlxup upgrades the firmware using embedded images. The latest mlxup executable and documentation are available inmlxup - Update and Query Utility .
  • Page 72 Status: Update required   Perform FW update? [y/N]: y Device # : Up to date Device # : Updating FW ... Done   Restart needed updates to take effect. Log File: /var/log/mlxup/mlxup-yyyymmdd.log NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...

  • Page 73: Specifications

    PCIe 3.0 x16 bus can supply a maximum bandwidth of 128Gb/s only (=16 * 8GT/s, including overhead), and therefore cannot support 200Gb/s when both network ports of MCX556A-ECAT run at 100Gb/s. Power Voltage: 12V Power Cable NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 74 EMC: CE / FCC / VCCI / ICES / RCM tory RoHS: RoHS compliant The ConnectX-5 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA Infini Band product.
  • Page 75 Voltage: 12V Power Cable Passive Cables 17.1W Typical Power Adapter Card Passive Cables 20.0W Power Maximum Power 1.5W Active Cables 23.3W 2.5W Active Cables 25.5W Maximum power available through QSFP28 port: 5W NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 76 EMC: CE / FCC / VCCI / ICES / RCM RoHS: RoHS compliant The ConnectX-5 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA Infini Band product.
  • Page 77 For the 3D Model of the card, please refer to http://www.nvidia.com/page/3d_models. Dual-port Adapter Card Auxiliary PCIe Connection Card Slim-Line SAS Cable (25cm) Slim-Line SAS Cable (35cm) Bracket Mechanical Drawing Adapter Card Tall Bracket Auxiliary PCIe Connection Card Tall Bracket NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...
  • Page 78 Adapter Card Tall Bracket Auxiliary PCIe Connection Card Tall Bracket NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...

  • Page 79: Monitoring

    IC thermal requirements. Adapter Card Heatsink The heatsink is attached to the ConnectX-5 IC to dissipate the heat from the ConnectX-5 IC. It is attached by spring-loaded screws. ConnectX-5 IC has a thermal shutdown safety mechanism that automatically shuts down the ConnectX-5 card in cases of high-temperature events, improper thermal coupling, or heatsink removal.

  • Page 80: Finding The Guid/Mac And Serial Number On The Adapter Card

    Serial Number on the Adapter Card Each NVIDIA adapter card has a different identifier printed on the label: serial number and the card MAC for the Ethernet protocol and the card GUID for the InfiniBand protocol. VPI cards have both a GUID and a MAC (derived from the GUID).
  • Page 81 Auxiliary PCIe Connection Card Board Label Slim-Line SAS Harness Board Label © Copyright 2023, NVIDIA. PDF Generated on 06/07/2024 NVIDIA ConnectX-5 InfiniBand/Ethernet Socket Direct Adapter Cards User Manual...

This manual is also suitable for:

Mcx556m-ecat-s25Mcx556m-ecat-s35a

Table of Contents