Download
Share
Print this page
Nvidia MCX4621A-XCAB User Manual
  1. Manuals
  2. Brands
  3. Nvidia Manuals
  4. Adapter
  5. MCX4621A-XCAB
  6. User manual
Nvidia MCX4621A-XCAB User Manual

Nvidia MCX4621A-XCAB User Manual

Connectx-4 lx ethernet adapter cards for ocp 3.0
Hide thumbs

Advertisement

Quick Links

Download this manual
NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP
3.0 User Manual

Advertisement

loading
Need help?

Need help?

Do you have a question about the MCX4621A-XCAB and is the answer not in the manual?

Questions and answers

Related Manuals for Nvidia MCX4621A-XCAB

Summary of Contents for Nvidia MCX4621A-XCAB

  • Page 1 NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 2: Table Of Contents

    Updating Adapter Firmware Updating Adapter Firmware Speci cations Speci cations Finding the MAC on the Adapter Card Finding the MAC on the Adapter Card Document Revision History Document Revision History NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 3 This manual is intended for the installer and user of these cards. The manual assumes basic familiarity with Ethernet network and architecture speci cations. Technical Support Customers who purchased NVIDIA products directly from NVIDIA are invited to contact us through the following methods: URL: https://www.nvidia.com > Support E-mail: enterprisesupport@nvidia.com...
  • Page 4 900-9X422- management, MCX4621A-XCAB 0052-SB0 10GbE Dual-port SFP28, PCIe3.0 x8, Thumbscrew bracket, ROHS Compliant 900-9X422- MCX4621A-ACAB ConnectX®-4 Lx EN 0053-SB0 network interface card for OCP 3.0, with host management, NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 5 Firmware rmware. See Update NVIDIA Firmware Utility (mlxup) Documentation. NVIDIA User Manual Firmware describing the Tools (MFT) set of MFT User Manual rmware management tools for a single NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 6 NVIDIA o ers one of the industry’s broadest portfolio of 40GbE, 56GbE, 100GbE, 200GbE and 400GbE cables, including Direct Attach NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 7 When discussing memory sizes, MB and MBytes are used in this document to mean size in MegaBytes. The use of Mb or Mbits (small b) indicates size in MegaBits. In this document, PCIe is used to mean PCI Express. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 8: Introduction

    1. ConnectX-4 Lx OCP 3.0 cards were tested for Shock & Vibe in accordance with NVIDIA speci cations and setups, as the OCP spec 3.0 available at that time did not contain any S&V de nitions. A newer version of the OCP spec 3.0 has de ned S&V speci cations and NVIDIA is in the midst of retesting these cards to comply with OCP spec 3.0.

  • Page 9: Features And Bene Ts

    • IEEE 802.3by, Ethernet Consortium25 • IEEE 802.3by 25 Gigabit Ethernet • IEEE 802.3ae 10 Gigabit Ethernet • IEEE 802.3ap based auto-negotiation and KR startup • Proprietary Ethernet protocols (20/40GBASE- R2, 50GBASE-R4) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 10 RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore signi cantly reduces application run time. ConnectX-4 Lx NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 11: Operating Systems/Distributions

    The adapter supports a Network Controller Sideband Interface (NC-SI), MCTP over SMBus NC-SI and MCTP over PCIe - Baseboard Management Controller interface. Wake-on-LAN (WoL) Supported Reset-on-Lan (RoL) Supported Operating Systems/Distributions RHEL/CentOS Windows FreeBSD NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 12 OpenFabrics Enterprise Distribution (OFED) OpenFabrics Windows Distribution (WinOF-2) Connectivity Interoperable with 1/10/25 Gb/s Ethernet switches Passive copper cable with ESD protection Powered connectors for optical and active cable support NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 13: Interfaces

    There are two, LED0 and LED1, and speed. LED0 is bi-color (yellow and green) LED and LED1 is a single color (green) LED. I/O LEDs per port to indicate link status. Link Indications NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 14: Smbus Interface

    ConnectX-4 Lx IC has a thermal shutdown safety mechanism that automatically shuts down the ConnectX-4 Lx card in cases of a high-temperature event, improper thermal coupling or heatsink removal. SMBus Interface NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 15: Voltage Regulators

    CPLD Interface The adapter card incorporates a CPLD device that controls the networking port LEDs and the scan chain. It draws its power supply from 3.3V_EDGE. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 16: Hardware Installation

    55°C (131°F). An air ow of 200LFM at this maximum ambient temperature is required for HCA cards and NICs. To guarantee proper air ow, allow at least NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 17: Installation Procedure Overview

    IEC 60 825-1:1993 + A1:1997 + A2:2001 and EN 60825-1:1994+A1:1996+ A2:20 Installation Procedure Overview The installation procedure of ConnectX-4 Lx adapter cards for OCP Spec 3.0 involves the following steps: NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 18: System Requirements

    System System Requirements Warning Unless otherwise speci ed, NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous and dust (particulate) contamination. The operating environment should meet severity level G1 as per ISA 71.04 for gaseous contamination and ISO 14644-1 class 8 for...

  • Page 19: Hardware Requirements

    Software Requirements See Operating Systems/Distributions section under the Introduction section. Software Stacks - NVIDIA OpenFabric software package MLNX_OFED for Linux, WinOF-2 for Windows, and VMware. See the Driver Installation section. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 20: Safety Precautions

    Unpack and remove the ConnectX-4 Lx card. Check the parts for visible damage that may have occurred during shipping. Please note that the cards must be placed on an antistatic surface. Warning NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 21 The required torx tool type as speci ed in the instructions. Removing the Existing Bracket 1. Using the torx tool type listed in the below table, remove the screws according to the instructions per OCP 3.0 bracket type. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 22 Bracket Bracke Bracket Bracket w) Bracket Warning Be careful not to put stress on the LEDs on the adapter card. 3. Save the two screws. Installing the New Bracket NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 23 1. Assemble the new bracket onto the card. Pull- Internal Pull-tab Internal Ejector- Ejector-Latch Lock (Thumbscr Lock Latch Bracket Brack Bracket ew) Bracket Bracket Bracket Warning Do not force the bracket onto the adapter card. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 24 Use the speci ed torx tool type and apply the speci ed torque on the screws per bracket form factor. Pull-tab Pull- Internal Ejector- Internal Ejector- (Thumbsc Lock Latch Lock Latch rew) Brack Bracket Bracket Bracket Bracket Bracket NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 25 The following gures are for illustration purposes only. 1. Before installing the card, make sure that the system is o and the power cord is not connected to the server. Please follow proper electrical grounding procedures. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 26 4. Push the card until connectors are in a full mate. Pull- Internal Pull-tab (Thum Ejector-Latch Internal Lock Ejector-Latch Lock (Thumbscre bscrew Bracket Bracket Bracket Bracket w) Bracket Bracke 5. Secure the card. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 27: Cables And Modules

    Close the screw once the ejector. clockwise connecto until rmly rs are in locked. a full mate. To uninstall the adapter card, see Uninstalling the Card. Cables and Modules NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 28 See Adapter Card LED Operations. 4. After plugging in a cable, lock the connector using the latching mechanism particular to the cable vendor. When a logical connection is made, Green LED1 will NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 29 1. Open Device Manager on the server. Click Start => Run, and then enter devmgmt.msc. 2. Expand System Devices and locate your NVIDIA ConnectX-4 Lx adapter card. 3. Right click the mouse on your adapter's row and select Properties to display the adapter card properties window.
  • Page 30 5. In the Value display box, check the elds VEN and DEV ( elds are separated by ‘&’). In the display example above, notice the sub-string “PCI\VEN_15B3&DEV_1003”: VEN is equal to 0x15B3 – this is the Vendor ID of NVIDIA; and DEV is equal to 1018 (for ConnectX-4 Lx) – this is a valid NVIDIA PCI Device ID.
  • Page 31 Pull-tab (Thumbscrew) Internal Lock Bracket Ejector-Latch Bracket Bracket While holding the heatsink, Rotate the captive Open the ejector latch. gently push the card out of screw the server. counterclockwise. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 32 Latch Bracket Bracket Bracket Bracket Brack While holding the Rotate the Open the heatsink, gently push captive ejector the card out of the screw latch. server. counterclo ckwise. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 33 Lock bscre Latch Bracket Bracket Bracket Bracket Brack Gently pull out the While Open the adapter card from the holding ejector server. the tab, latch. pull out adapter card. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 34 Pull- Pull-tab Ejector- Internal (Thum Ejector- (Thumbscr Internal Lock Bracket Latch Lock bscre Latch Bracket Bracket Bracket Bracket Brack NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 35: Driver Installation

    The following example shows a system with an installed adapter card: # lspci -v | grep Mellanox 86:00.0 Network controller [0207]: Mellanox Technologies MT27710 Family Subsystem: Mellanox Technologies Device 0014 NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 36 SHA256 MLNX_OFED_LINUX-<ver>-<OS label>.iso Installing MLNX_OFED Installation Script The installation script, mlnxofedinstall, performs the following: Discovers the currently installed kernel NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 37 Usage ./mnt/mlnxofedinstall [OPTIONS] The installation script removes all previously installed OFED packages and re-installs from scratch. You will be prompted to acknowledge the deletion of the old packages. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 38 ConnectX card, not regenerating the initramfs may even cause the system to fail to reboot. In such cases, the installer will recommend running the following command to update the initramfs: NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 39 Example: The following command will create a MLNX_OFED_LINUX ISO image for RedHat 7.3 under the /tmp directory. # ./MLNX_OFED_LINUX-x.x-x-rhel7. x86_64/mlnx_add_kernel_support.sh -m /tmp/MLNX_OFED_LINUX-x.x-x-rhel7. -x86_64/ --make-tgz Note: This program will create MLNX_OFED_LINUX TGZ rhel7. under /tmp directory. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 40 For the list of installation options, run: ./mlnxofedinstall --h Installation Procedure This section describes the installation procedure of MLNX_OFED on NVIDIA adapter cards. 1. Log in to the installation machine as root. 2. Mount the ISO image on your machine.
  • Page 41 Installation finished successfully. Attempting to perform Firmware update... Querying Mellanox devices firmware ... Note For unattended installation, use the --force installation option while running the MLNX_OFED installation script: /mnt/mlnxofedinstall --force Note NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 42 Device #1: ---------- Device Type: ConnectX-X Part Number: MCXXXX-XXX PSID: MT_ PCI Device Name: 0b:00.0 Base MAC: 0000e41d2d5cf810 Versions: Current Available FW XX.XX.XXXX Status: Up to date NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 43 To obtain rmware for this device, please contact your HW vendor. 4. Case A: If the installation script has performed a rmware update on your network adapter, you need to either restart the driver or reboot your system NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 44 /etc/in niband/info. Most of the OFED components can be con gured or recon gured after the installation, by modifying the relevant con guration les. See the relevant chapters in this manual for details. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 45 Socket Direct Cards Software Most of MLNX_OFED packages are installed under the “/usr” directory except for the following packages which are installed under the “/opt” directory: fca and ibutils NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 46 - r`/updat es on SLES Fedora Distribu tions /lib/mod ules/`un ame - r`/extra/ mlnx- ofa_ker nel on RHEL other RedHat like Distribu tions /lib/mod ules/`un ame - r`/updat NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 47 ‘- without- update’ rmwar version of the adapter device is older than the rmwar version included with the NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 48 Expansi on ROM image. In case your machine has unsupported network adapter device, no rmware update will occur and the error message below will be printed. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 49 Driver Load Upon System Boot Upon system boot, the NVIDIA drivers will be loaded automatically. To prevent the automatic load of the NVIDIA drivers upon system boot: 1. Add the following lines to the "/etc/modprobe.d/mlnx.conf" le. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 50 (Hard Reset) Code Reset) Standard ConnectX- 4/Connect X-4 Lx or higher Adapters with Multi- Host Support Socket Direct Cards Software Most MLNX_ OFED packag es are installe d under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 51 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) “/usr” directo except for the followin packag which installe d under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 52 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel module s are installe d under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 53 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 54 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 55 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwar e of existing networ adapter devices NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 56 Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) will be update d if the followin g two conditi ons are ful lled NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 57 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 58 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 59 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ’s NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 60 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 61 "The rmwar e for this device is not distribu inside NVIDIA driver: 0000:0 NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 62 No rmware was found for the adapter device Invalid parameter Not enough free space applicable to this system con guratio n. This can occur when NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 63 Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) Standard ConnectX- 4/Connect X-4 Lx or higher Adapters with Multi- Host Support Socket Direct Cards NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 64 Adapter Restart (Hard Reset) Code Reset) Software Most MLNX_ OFED packag es are installe d under “/usr” directo except for the followin packag which installe d under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 65 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel module s are installe d under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 66 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 67 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 68 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 69 Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwar e of existing networ adapter devices will be update d if the followin g two conditi ons are ful lled NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 70 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 71 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 72 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ’s NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 73 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 74 Reset) In case your machin e has unsupp orted networ adapter device, rmwar update will occur and the error messag e below will be printed. "The rmwar e for NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 75 1:00.0 (PSID: IBM215 011003 obtain rmwar e for this device, please contact your vendor." Installation ended successfully installation failed No rmware was found for the adapter device Invalid parameter NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 76 Failed to start the mst driver Software Most MLNX_ OFED packag es are installe d under “/usr” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 77 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) except for the followin packag which installe d under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 78 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel module s are installe d under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 79 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 80 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 81 Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwar e of existing networ adapter devices will be update d if the followin g two NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 82 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) conditi ons are ful lled NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 83 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 84 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 85 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ’s NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 86 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 87 "The rmwar e for this device is not distribu inside NVIDIA driver: 0000:0 1:00.0 (PSID: IBM215 011003 NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 88 RPM. Additional Installation Procedures Installing MLNX_OFED Using YUM This type of installation is applicable to RedHat/OL and Fedora operating systems. Setting up MLNX_OFED YUM Repository NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 89 1354 Saving to: ?RPM-GPG-KEY-Mellanox?   %[=================================================>] --.-K/s in 0s     MB/s) - ?RPM-GPG-KEY-Mellanox? 2018 saved [ 1354 1354 4. Install the key. # sudo rpm -- RPM-GPG-KEY-Mellanox import NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 90 [mlnx_ofed] name=MLNX_OFED Repository baseurl=file: ///<path to extracted MLNX_OFED package>/RPMS enabled= gpgkey=file: ///<path to the downloaded key RPM-GPG-KEY-Mellanox> gpgcheck= 7. Check that the repository was successfully added. # yum repolist NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 91 -kmp> -k $(uname -r) -m /mnt/ Note: This program will create MLNX_OFED_LINUX TGZ rhel7. under /tmp directory. Do you want to ?[y/N]:y continue See log file /tmp/mlnx_iso.4120_logs/mlnx_ofed_iso. .log 4120   NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 92 "/etc/yum.repos.d/mlnx_ofed.repo" with the following content: [mlnx_ofed] name=MLNX_OFED Repository baseurl=file: ///<path to extracted MLNX_OFED package>/RPMS enabled= gpgcheck= 6. Check that the repository was successfully added. # yum repolist Loaded plugins: product-id, security, subscription- manager NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 93 : MLNX_OFED bluefield installer (with KMP support) package mlnx-ofed-bluefield-user-only.noarch : MLNX_OFED bluefield-user-only installer (User Space package packages only) mlnx-ofed-dpdk.noarch : MLNX_OFED dpdk installer package (with KMP support) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 94 : MLNX_OFED vma-eth- user-only installer (User Space packages only) package mlnx-ofed-vma-user-only.noarch : MLNX_OFED vma-user-only installer (User Space packages only) package mlnx-ofed-vma-vpi.noarch : MLNX_OFED vma-vpi installer (with KMP support) package NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 95 4/Connec tX-4 Lx or higher Adapters with Multi- Host Support Socket Direct Cards Software Most MLNX_ OFED packag es are installe under “/usr” directo except for the followi packag NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 96 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) which installe under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 97 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel modul es are installe under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 98 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 99 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 100 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 101 Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwa re of existin networ adapte device s will update d if the followi ng two conditi ons are ful lle NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 102 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 103 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 104 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 105 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) r’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 106 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 107 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 108 "The rmwa re for this device is not distrib uted inside NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 109 " Installation ended successfully installation failed No rmware was found for the adapter device Invalid parameter Not enough free space applicable to NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 110 Failed to start the mst driver Software Most MLNX_ OFED packag es are installe under “/usr” directo except for the NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 111 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) followi packag which installe under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 112 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel modul es are installe under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 113 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 114 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 115 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 116 Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwa re of existin networ adapte device s will update d if the followi ng two conditi NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 117 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ons are ful lle NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 118 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 119 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 120 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) r’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 121 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 122 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 123 In case your machin e has unsup ported networ adapte device, rmwa update will occur error messa below will be printed "The rmwa re for this device is not NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 124 " Installs all available mlnx- packages in ofed-all MLNX_OFE Installs basic mlnx- packages ofed- required for basic running NVIDIA cards Installs mlnx- packages ofed- required by guest guest OS NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 125 VMA to support VPI Installs packages blue eld required for BlueField Installs packages dpdk required for DPDK Installs dpdk- packages upstream required for -libs DPDK using RDMA-Core NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 126 .fc21.x86_64.noarch : 3.17 MLNX_OFED hypervisor installer   kernel package 3.17 .fc21.x86_64 (without KMP support) mlnx-ofed-vma- .fc21.x86_64.noarch : MLNX_OFED vma 3.17 installer   kernel .fc21.x86_64 (without package 3.17 KMP support) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 127 Please note that the resulting MLNX_OFED repository will contain unsigned RPMs, therefore, you should set 'gpgcheck=0' in the repository con guration 2. Install the desired group. # yum install mlnx-ofed-all Loaded plugins: langpacks, product-id, subscription- manager Resolving Dependencies NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 128 This type of installation is applicable to Debian and Ubuntu operating systems. Setting up MLNX_OFED apt-get Repository 1. Log into the installation machine as root. 2. Extract the MLNX_OFED package on a shared location in your network. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 129 Setting up MLNX_OFED apt-get Repository Using --add-kernel-support 1. Log into the installation machine as root. 2. Mount the ISO image on your machine and copy its content to a shared location in your network. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 130 WARNING: by setting in the repository conf file. 'gpgcheck=0' Created /tmp/MLNX_OFED_LINUX- -rhel7. -x86_64- ext.tgz 4. Open the tarball. # cd /tmp/ # tar -xvf /tmp/MLNX_OFED_LINUX- -rhel7. -x86_64- ext.tgz NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 131 (with DKMS support) mlnx-ofed-kernel-only - MLNX_OFED kernel-only installer (with DKMS support) package mlnx-ofed-bluefield - MLNX_OFED bluefield installer (with DKMS support) package mlnx-ofed-hpc-user-only - MLNX_OFED hpc-user-only installer (User Space packages only) package NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 132 - MLNX_OFED basic-user-only installer (User Space packages only) package mlnx-ofed-basic-exact - MLNX_OFED basic installer package (with DKMS support) (exact) mlnx-ofed-basic - MLNX_OFED basic installer (with package DKMS support) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 133 4/Connec tX-4 Lx or higher Adapters with Multi- Host Support Socket Direct Cards Software Most MLNX_ OFED packag es are installe under “/usr” directo except for the followi packag NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 134 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) which installe under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 135 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel modul es are installe under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 136 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 137 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 138 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 139 Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwa re of existin networ adapte device s will update d if the followi ng two conditi ons are ful lle NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 140 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 141 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 142 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 143 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) r’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 144 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 145 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 146 "The rmwa re for this device is not distrib uted inside NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 147 " Installation ended successfully installation failed No rmware was found for the adapter device Invalid parameter Not enough free space applicable to NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 148 Failed to start the mst driver Software Most MLNX_ OFED packag es are installe under “/usr” directo except for the NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 149 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) followi packag which installe under “/opt” directo NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 150 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) kernel modul es are installe under NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 151 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 152 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 153 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 154 Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) Firmware rmwa re of existin networ adapte device s will update d if the followi ng two conditi NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 155 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ons are ful lle NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 156 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) ‘-- e’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 157 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 158 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) r’ NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 159 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 160 Standard Action \ Driver Cold Reboot Return Reboot (Soft Meaning Adapter Restart (Hard Reset) Code Reset) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 161 In case your machin e has unsup ported networ adapte device, rmwa update will occur error messa below will be printed "The rmwa re for this device is not NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 162 " Installs all available mlnx- packages in ofed-all MLNX_OFE Installs basic mlnx- packages ofed- required for basic running NVIDIA cards Installs mlnx- packages ofed- required by guest guest OS NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 163 VMA to support VPI Installs packages blue eld required for BlueField Installs packages dpdk required for DPDK Installs dpdk- packages upstream required for -libs DPDK using RDMA-Core NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 164 MLNX_OFE mlnx- D vma-eth ofed- installer vma-eth package MLNX_OFE mlnx- D vma-vpi ofed- installer vma-vpi package MLNX_OFE D DKMS knem- support for dkms mlnx-ofed kernel modules NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 165 MLNX_OFE mlnx- D mlnx- ofed- ofed-basic- basic- exact exact installer package MLNX_OFE dpdk- D dpdk- upstream upstream- -libs libs installer package 2. Install the desired group. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 166: Vmware Driver Installation

    Performance Tuning Guide for NVIDIA Network Adapters. VMware Driver Installation This section describes VMware Driver Installation. Hardware and Software Requirements NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 167 #> esxcli software vib install –d <path>/<bundle_file> Example: #> esxcli software vib install -d /tmp/MLNX-NATIVE-ESX- ConnectX- -5_4. -10EM- .zipesxcli 16.8 650.0 0.4240417 3. Reboot the machine. 4. Verify the driver was installed successfully. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 168 2. List all the existing NATIVE ESXi driver modules. (See Step 4 in Installing NATIVE ESXi Driver for VMware vSphere.) 3. Remove each module: #> esxcli software vib remove -n nmlx5-rdma #> esxcli software vib remove -n nmlx5-core NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 169: Firmware Programming

    Windows Driver Installation For Windows, download and install the latest WinOF-2 for Windows software package available via the NVIDIA website at: WinOF-2 webpage. Follow the installation instructions included in the download package (also available from the download page). The snapshots in the following sections are presented for illustration purposes only. The installation interface may slightly vary, depending on the operating system in use.
  • Page 170 2. Open a CMD console (Click Task Manager-->File --> Run new task and enter CMD). 3. Enter the following command. echo %PROCESSOR_ARCHITECTURE% Note On an x64 (64-bit) machine, the output will be “AMD64”. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 171: Attended Installation

    2. Go to the WinOF-2 web page at: https://www.nvidia.com/en-us/networking/ > Products > Software > In niBand Drivers (Learn More) > Nvidia WinOF-2. 3. Download the .exe image according to the architecture of your machine (see Step 1). The name of the .exe is in the following format: MLNX_WinOF2- <version>_<arch>.exe.
  • Page 172 MlxRshim drivers Installed, Please remove the following oem inf les from driver store: <oem inf list>" 5. [Optional] If you want to skip the check for unsupported devices, run. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 173 MLNX_WinOF2_<revision_version>_All_Arch.exe /v " SKIPUNSUPPORTEDDEVCHECK=1" 6. Click Next in the Welcome screen. 7. Read and accept the license agreement and click Next. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 174 If the user has an OEM card. In this case, the rmware will not be displayed. If the user has a standard NVIDIA® card with an older rmware version, the rmware will be updated accordingly. However, if the user has both an OEM...
  • Page 175 NVIDIA® card, only the NVIDIA® card will be updated. 10. Select a Complete or Custom installation, follow Step a onward. 1. Select the desired feature to install: Performances tools - install the performance tools that are used to measure performance in user environment NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 176 Management tools - installation tools used for management, such as mlxstat Diagnostic Tools - installation tools used for diagnostics, such as mlx5cmd 2. Click Next to install the desired tools. 11. Click Install to start the installation. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 177 12. In case rmware upgrade option was checked in Step 7, you will be noti ed if a rmware upgrade is required (see 13. Click Finish to complete the installation. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 178: Unattended Installation

    The following is an example of an unattended installation session. 1. Open a CMD console-> Click Start-> Task Manager File-> Run new task-> and enter CMD. 2. Install the driver. Run: NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 179 5. [Optional] If you do not wish to upgrade your rmware version (i.e., MT_SKIPFWUPGRD default value is False ). MLNX_WinOF2-[Driver/Version]_<revision_version>_All_Arch.exe /vMT_SKIPFWUPGRD= 6. [Optional] If you do not want to install the Rshim driver, run. MLNX_WinOF2_<revision_version>_All_Arch.exe /v " MT_DISABLE_RSHIM_INSTALL=1" Note NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 180: Firmware Upgrade

    SKIPUNSUPPORTEDDEVCHECK=1" Firmware Upgrade If the machine has a standard NVIDIA® card with an older rmware version, the rmware will be automatically updated as part of the NVIDIA® WinOF-2 package installation. For information on how to upgrade rmware manually, please refer to MFT User Manual.

  • Page 181: Updating Adapter Firmware

    To check that your card is programmed with the latest available rmware version, download the mlxup rmware update and query utility. The utility can query for available NVIDIA adapters and indicate which adapters require a rmware update. If the user con rms, mlxup upgrades the rmware using embedded images. The latest mlxup executable and documentation are available in mlxup - Update and Query Utility .
  • Page 182 Up to date   Perform FW update? [y/N]: y Device #1: Up to date Device #2: Updating FW ... Done   Restart needed for updates to take effect. Log File: /var/log/mlxup/mlxup-yyyymmdd.log NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 183: Speci Cations

    25mW 6.35W Cables Maximum power available through SFP28 port: 1.5W Air ow Port Heatsi Heatsink to Port Cable Type Cold- Hot-Aisle (55°C Ambient) Aisle (35°C Ambie Passive Cables 150LFM NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 184 10GBASE-SR, 10GBASE-LR,10GBASE-ER, 10GBASE-CX4, 10GBASE-CR, 10GBASE-KR, SGMII Protocol Support Data Rate: 1/10/25 Gb/s Ethernet PCI Express Gen3: SERDES @ 8.0GT/s, 8 lanes (2.0 and 1.1 compatible) Power and Voltage: 3.3V_EDGE, 12V_EDGE Air ow NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...
  • Page 185 10% to 85% relative humidity Humidity ntal Non-operational 10% to 90% relative humidity Altitude 3050m (Operational) Regulatory Safety CB / cTUVus / CE CE / FCC / VCCI / ICES / RCM NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 186: Board Mechanical Drawing And Dimensions

    The non-operational storage temperature speci cations apply to the product without its package. Board Mechanical Drawing and Dimensions Note All dimensions are in millimeters. PCB mechanical tolerance is +/- 0.13mm. NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 187: Finding The Mac On The Adapter Card

    Finding the MAC on the Adapter Card Each NVIDIA adapter card has a di erent identi er printed on the label: serial number and the card MAC for the Ethernet protocol. Note The product revisions indicated on the labels in the following gures do not necessarily represent the latest revisions of the cards.
  • Page 188 MCX4621A-ACAB Board Label (Example) NVIDIA ConnectX-4 Lx Ethernet Adapter Cards for OCP 3.0 User Manual...

  • Page 189: Document Revision History

    NVIDIA shall have no liability for the consequences or use of such information or for any infringement of patents or other rights of third parties that may...
  • Page 190 NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.<br/><br/>No license, either expressed or implied, is granted under any NVIDIA patent right, copyright,...

This manual is also suitable for:

Mcx4621a-acab900-9x422-0052-sb0900-9x422-0053-sb0