Nvidia BlueField-2 BF2500 User Manual
  1. Manuals
  2. Brands
  3. Nvidia Manuals
  4. PCI Card
  5. BlueField-2 BF2500
  6. User manual
Nvidia BlueField-2 BF2500 User Manual

Nvidia BlueField-2 BF2500 User Manual

Infiniband/ethernet dpu controller
Hide thumbs Also See for BlueField-2 BF2500:
Table of Contents

Advertisement

Quick Links

Download this manual
NVIDIA BlueField-2 BF2500 InfiniBand/Ethernet DPU Controller
User Manual
Exported on Oct/31/2023 10:43 AM

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the BlueField-2 BF2500 and is the answer not in the manual?

Questions and answers

Subscribe to Our Youtube Channel

Related Manuals for Nvidia BlueField-2 BF2500

Summary of Contents for Nvidia BlueField-2 BF2500

  • Page 1 NVIDIA BlueField-2 BF2500 InfiniBand/Ethernet DPU Controller User Manual Exported on Oct/31/2023 10:43 AM...

  • Page 2: Table Of Contents

    Table of Contents Introduction............................10 System Requirements Overview ..........................10 Package Contents ...............................10 Card Package ..............................11 Accessories Kit .............................. 11 Ideal Solution for JBOF and JBOD Systems........................12 Features and Benefits ............................13 Supported Interfaces ..........................16 Layout and Connector Information...........................16 BF2500 DPU Controller - Component Side ......................16 BF2500 DPU Controller - Print Side........................
  • Page 3 GbE OOB Management ............................. 21 1GbE OOB Management LEDs Interface......................21 NC-SI Management Interface ..........................22 UART Interface Connectivity .......................... 22 Mini USB Type B Connector ..........................23 External PCIe Power Supply Interfaces ........................23 eMMC Interface.............................. 24 Networking LED Interfaces ..........................24 RTC Battery ..............................
  • Page 4 Cables and Cabling Configurations ........................ 49 RJ45 Cable for the 1GbE OOB Management Interface.....................49 USB Interface Cabling ............................49 Thermal Sensors............................51 Hardware Installation ..........................52 Safety Warnings ..............................52 Installation Procedure Overview..........................52 System Requirements ............................53 Hardware Requirements ........................... 53 Airflow Requirements ............................53 Software Requirements ............................
  • Page 5 Installing Linux on BlueField Controller Card.......................59 Software Requirements ............................ 60 Preparing Host-Side Environment ........................60 Setup Procedure With Installation Script......................61 Setup Procedure Without Installation Script......................61 Flashing BlueField Controller Card Bootloader Code ....................65 Opening Terminal Connection to BlueField Controller Card..................66 Using Initial Install Bootstream ........................
  • Page 6 MBF2H515B-HE[N/E]OT Specifications........................83 BlueField-2 BF2500 Mechanical Drawing and Dimensions....................84 Bracket Mechanical Drawing and Dimensions......................85 Finding the GUID/MAC and Serial Number on the Card ..................86 Document Revision History ......................... 87    ...
  • Page 7 About This Manual This User Manual describes NVIDIA® BlueField®-2 BF2500 DPU Controller card. It provides details as to the interfaces of the board, specifications, required software and firmware for operating the card, hardware installation, driver installation and bring-up instructions. EOL'ed (End of Life) Ordering Part Numbers...
  • Page 8 Copper cables (DACs), copper splitter cables, Active Optical Cables (AOCs) and transceivers in a wide range of lengths from 0.5m to 10km. In addition to meeting IBTA standards, NVIDIA tests every product in an end-to-end environment ensuring a Bit Error Rate of less than 1E-15. ...
  • Page 9 InfiniBand Trade Association (IBTA) InfiniBand® specification Release 1.3.1, November 2, 2016 and Vol. 2, Release 1.4 , InfiniBand Architecture Specification  and Vol 2 - Release 1.5. NVIDIA DOCA SDK software. DOCA SDK Software Documentation  Document Conventions When discussing memory sizes, MB and MBytes are used in this document to mean size in mega Bytes. The use of Mb or Mbits (small b) indicates size in mega bits.

  • Page 10: Introduction

    This is the User Manual for Technologies BlueField®-2 BF2500 DPU Controller. This document provides details as to the interfaces of the card, specifications, required software and firmware for operating the board, and a step-by-step plan of how to bring up the BlueField-2 BF2500 DPU Controller. ...

  • Page 11: Card Package

    Product Overview BlueField-2 BF2500 DPU Controller features the second generation BlueField-2 I/O Processing Unit (IPU) – an innovative and high-performance programmable networking engine. The IPU integrates an array of eight powerful 64-bit Armv8 A72  cores interconnected by a coherent mesh with a DDR4 memory controller and a dual-port Ethernet network controller. Providing unmatched scalability and efficiency, the BF2500 DPU Controller is the ideal...

  • Page 12: Ideal Solution For Jbof And Jbod Systems

     The BlueField BF2500 DPU Controller should be installed only in a JBOF and JBOD Systems as it functions as a PCIe root-complex (RC) initiating PCIe bus operations. Installing it in a regular host system may damage the card. Ideal Solution for JBOF and JBOD Systems  BlueField-2 I/O Processing Unit (IPU) is a highly integrated and efficient controller, optimized for NVMe storage systems, Network Functions Virtualization (NFV), Cloud, and Machine Learning workloads.

  • Page 13: Features And Benefits

    Part Number MBF2H516B-EEEOT MBF2H516B-EENOT MBF2H515B-HEEOT MBF2H515B-HENOT RoHS RoHS Compliant BlueField SoC BlueField-2 P-Series 8 cores (high-bin) a. Note: Refer to BlueField-2 Software and Firmware release notes for the availability of PCIe Gen 4.0 capabilities. For more detailed information see Specifications.  Features and Benefits This section describes hardware features and capabilities.
  • Page 14 Feature Description - IEEE 802.3ba 40 Gigabit Ethernet - IEEE 802.3by 25 Gigabit Ethernet - IEEE 802.3ae 10 Gigabit Ethernet - IEEE 802.3ap based auto-negotiation and KR startup - IEEE 802.3ad, 802.1AX Link Aggregation - IEEE 802.1Q, 802.1P VLAN tags and priority - IEEE 802.1Qau (QCN) - Congestion Notification - IEEE 802.1Qaz (ETS)
  • Page 15 Feature Description Storage A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can Acceleration leverage RDMA for high-performance storage access. • NVMe over Fabric offloads for the target machine • T10-DIF Signature Handover BlueField-2 SmartNIC may operate as a co-processor offloading specific storage tasks from the host, isolating part of the storage media from the host, or enabling abstraction of software-defined storage logic using the BlueField-2 Arm cores.

  • Page 16: Supported Interfaces

    Supported Interfaces Layout and Connector Information The BF2500 DPU Controller is a FHHL card. The component height on the top and bottom of the card complies with the PCIe specification. The below figures show the connector and LED designations.  The below figures are for illustration purposes only and might not reflect the current revision of the BF2500 card. BF2500 DPU Controller - Component Side...

  • Page 17: Bf2500 Dpu Controller - Print Side

    BF2500 DPU Controller - Print Side...
  • Page 18 Callou Interface Description BlueField-2 Data Processing Unit (DPU) BlueField-2 DPU with heatsink PCIe x16 Gen 3.0/4.0 Edge Connector The interface between the BlueField-2 Controller and the JBOF system. InfiniBand QSFP56 Interfaces The network ports of the ConnectX®-6 adapter cards are compliant with the InfiniBand Architecture Specification, Release 1.3. InfiniBand traffic is transmitted through the cards' QSFP56 connectors.

  • Page 19: Interfaces Detailed Description

    Applies to Crypto enabled OPNs. BlueField-2 BF2500 DPU Controller addresses the concerns of modern data centers by combining hardware encryption accelerators with embedded software and fully integrated advanced network capabilities, making it an ideal platform for developing proprietary security applications. It enables a distributed security architecture by isolating and protecting each individual workload and providing flexible control and visibility at the server and workload level, controlling risk at the server access layer.

  • Page 20: Ethernet Qsfp56 Interfaces

    Ethernet QSFP56 Interfaces The network ports of the BlueField-2 BF2500 DPU Controller are compliant with the IEEE 802.3 Ethernet standards listed in  Ethernet traffic is transmitted through the cards' QSFP56 connectors. QSFP56 Port Over-current Protection The power consumption for each QSFP56 module is controlled by a 5W current limiting power distribution switch. Every power limiting switch is controlled by a signal, which is enabled by default.

  • Page 21: Gbe Oob Management

    GbE OOB Management The BlueField-2 BF2500 DPU Controller incorporates a 1GbE RJ45 out-of-band port that allows the network operator to establish trust boundaries in accessing the management function to apply it to network resources. It can also be used to ensure management connectivity (including the ability to determine the status of any network component) independent of the status of other in-band network components.

  • Page 22: Nc-Si Management Interface

    NC-SI Management Interface The Network Controller Sideband Interface (NC-SI) enables the connection of a Baseboard Management Controller (BMC) to a card for the purpose of enabling out-of-band remote manageability. NC-SI over the RMII interface is routed through the RMII connector with a flat cable.  The UART0 debug interface is connected to the NC-SI connector through the following pins: •...

  • Page 23: Mini Usb Type B Connector

    Mini USB Type B Connector The BlueField-2 BF2500 DPU Controller uses a Mini USB 2.0 Type B device. The purpose of the USB connector is debugging and loading new versions of the operating systems and firmware. External PCIe Power Supply Interfaces The BlueField-2 BF2500 DPU Controller consumes power from two power sources;...

  • Page 24: Emmc Interface

    The use of an external power supply interface is restricted to the ATX 6-pin PCIe power connector only. eMMC Interface The BlueField-2 BF2500 DPU Controller incorporates an eMMC interface on the card's print side. The eMMC is 16GB x8 NAND flash and is used for Arm boot, operating system storage and disk space.

  • Page 25: Rtc Battery

    Solid green Indicates a valid link with no active traffic Blinking green Indicates a valid link with active traffic LED1 and LED2 Link Status Indications - InfiniBand Protocol: LED Color and State Description A link has not been established Beacon command for locating the adapter card 1Hz blinking Yellow ...

  • Page 26: Emmc Interface

    eMMC Interface The BlueField-2 DPU incorporates an eMMC interface on the card's print side. The eMMC is 16GbE x8 NAND flash and is used for Arm boot, operating system storage and disk space.

  • Page 27: Pin Description

    Pin Description PCI Express Interface PCIe x16 Gen 4.0 Edge Connector The BF2500 Controller Card uses a PCI Express x16 connector (component and print sides) according to the PCI Express 4.0 specifications. The following two tables provide component side pins description and print side pins description. For further details, please refer to PCIe x16 Gen 3.0/4.0 Edge Interface.
  • Page 28 Pin# Pin Name Pin Definition Description OUTPUT Not connected 3.3V POWER 3.3V power 3.3V POWER 3.3V power PERST# OUTPUT PCIe Interface Reset#, drived by SoC Ground REFCLK+ OUTPUT PCIe Reference Differential Clock. Not connected (default) REFCLK- OUTPUT PCIe Reference Differential Clock. Not connected (default) Ground PERP0 OUTPUT...
  • Page 29 Pin# Pin Name Pin Definition Description PERN1 OUTPUT SoC TX Ground Ground PERP2 OUTPUT SoC TX PERN2 OUTPUT SoC TX Ground Ground PERP3 OUTPUT SoC TX PERN3 OUTPUT SoC TX Ground RSVD Not connected, option for SER_CLK RSVD Not connected, option for SER_CAP Ground PERP4 OUTPUT...
  • Page 30 Pin# Pin Name Pin Definition Description Ground PERP5 OUTPUT SoC TX PERN5 OUTPUT SoC TX Ground Ground PERP6 OUTPUT SoC TX PERN6 OUTPUT SoC TX Ground Ground PERP7 OUTPUT SoC TX PERN7 OUTPUT SoC TX Ground RSVD Not connected Ground PERP8 OUTPUT SoC TX...
  • Page 31 Pin# Pin Name Pin Definition Description Ground Ground PERP9 OUTPUT SoC TX PERN9 OUTPUT SoC TX Ground Ground PERP10 OUTPUT SoC TX PERN10 OUTPUT SoC TX Ground Ground PERP11 OUTPUT SoC TX PERN11 OUTPUT SoC TX Ground Ground PERP12 OUTPUT SoC TX PERN12 OUTPUT...
  • Page 32 Pin# Pin Name Pin Definition Description Ground Ground PERP13 OUTPUT SoC TX PERN13 OUTPUT SoC TX Ground Ground PERP14 OUTPUT SoC TX PERN14 OUTPUT SoC TX Ground Ground PERP15 OUTPUT SoC TX PERN15 OUTPUT SoC TX Ground...

  • Page 33: Side B

    Side B Pin# Pin Name Pin Definition Description POWER 12V power POWER 12V power POWER 12V power Ground SMCLK SMBUS_SCL - connected to BlueField IPU SMDAT SMBUS_SDA - connected to BlueField IPU Ground 3.3V POWER 3.3V power TRST# Not connected 3.3V_AUX POWER 3.3V auxilary power...
  • Page 34 Pin# Pin Name Pin Definition Description PETP0 SoC RX PETN0 SoC RX Ground PRESNT2# Not connected Ground PETP1 SoC RX PETN1 SoC RX Ground Ground PETP2 SoC RX PETN2 SoC RX Ground Ground PETP3 SoC RX PETN3 SoC RX Ground...
  • Page 35 Pin# Pin Name Pin Definition Description RSVD Not connected PRESNT2# Not connected Ground PETP4 SoC RX PETN4 SoC RX Ground Ground PETP5 SoC RX PETN5 SoC RX Ground Ground PETP6 SoC RX PETN6 SoC RX Ground Ground PETP7 SoC RX...
  • Page 36 Pin# Pin Name Pin Definition Description PETN7 SoC RX Ground PRESNT2# Not connected Ground PETP8 SoC RX PETN8 SoC RX Ground Ground PETP9 SoC RX PETN9 SoC RX Ground Ground PETP10 SoC RX PETN10 SoC RX Ground Ground...
  • Page 37 Pin# Pin Name Pin Definition Description PETP11 SoC RX PETN11 SoC RX Ground Ground PETP12 SoC RX PETN12 SoC RX Ground Ground PETP13 SoC RX PETN13 SoC RX Ground Ground PETP14 SoC RX PETN14 SoC RX Ground Ground...

  • Page 38: Power Sequencing

    Pin# Pin Name Pin Definition Description PETP15 SoC RX PETN15 SoC RX Ground PRESNT2# X16 PCIe card detection, connected to PRESNT1# RSVD Not connected Power Sequencing The BF2500 Controller Card requires the 12V ATX PCIe power to come up prior or at the same time to the +12V at the PCIe edge connector. NC-SI Management Interface The below table lists the NCSI Management Interface pins description.
  • Page 43 NC ...
  • Page 45 NC  ta  ta ...

  • Page 46: External Power Supply Interfaces

    External Power Supply Interfaces The below table lists the External Power Supply pins of the external power supply interface on the BF2500 Controller card. For further details, please refer to External PCIe Power Supply Interfaces.  The mechanical pinout of the 6-pin external +12V power connector is shown below. This +12V connector is sometimes referred to as EPS-12V. Care should be taken to ensure the power is applied to the correct pins as some 6-pin ATX type connector can have different pinouts.

  • Page 47: External +12V Mechanical Pinout Diagram

     External +12V Mechanical Pinout Diagram   UART Interface The below table lists the UART0 interface pins from the BlueField Controller Card perspective. For further details, please refer to UART0 Interface.  The electrical levels of the UART0 interface is TTL (3.3V).

  • Page 48: Mini Usb Interface

    Mini USB Interface The below table lists the Mini USB interface pins. For further details, please refer to Mini USB Interface. Name Description Data - Data + Ground...

  • Page 49: Cables And Cabling Configurations

    Cables and Cabling Configurations RJ45 Cable for the 1GbE OOB Management Interface USB Interface Cabling Debugging and loading new versions of the operating systems and firmware requires the use of Type A to Type B USB 2.0 Cable. The following figure shows an example of Type A to Type V USB Cable.

  • Page 51: Thermal Sensors

    Thermal Sensors The BlueField-2 BF2500 DPU Controller incorporates the BlueField-2 IPU which operates in the range of temperatures between 0C and 105C. There are three thermal threshold definitions for the BlueField-2 IPU which impact the overall system operation state: •...

  • Page 52: Hardware Installation

    Hardware Installation Installation and initialization of the BlueField-2 BF2500 DPU Controller require attention to the mechanical, power, and precautions for rack-mounted equipment. Safety Warnings  Safety warnings are provided here in the English language. For safety warnings in other languages, refer to the DPU Controller Installation Safety Instructions.

  • Page 53: System Requirements

    Hardware Requirements  Unless otherwise specified, NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous and dust (particulate) contamination. The operating environment should meet severity level G1 as per ISA 71.04 for gaseous contamination and ISO 14644-1 class 8 for cleanliness level.

  • Page 54: Software Requirements

    Software Requirements • Software Stacks - The BF2500 DPU Controller is shipped with Linux based Operating System burned on it which includes all needed drivers. For more information, please refer to the BlueField-2 Software User Manual. • Check System Requirements Overview under Introduction for more details.

  • Page 55: Installation Instructions

    This section provides detailed instructions on how to install your BlueField-2 2500 DPU Controller in a system.  The BlueField-2 BF2500 DPU Controller should be installed only in a JBOF System as it functions as a PCIe root-complex (RC) initiating PCIe bus operations. Installing it in a regular host system may damage the card.
  • Page 56 Insert the connector into the adapter card. Be careful to insert the connector straight into the cage. Do not apply any torque, up or down, to the connector cage in the adapter card. Verify that the connector locks in place.   When installing cables make sure that the latches engage.

  • Page 57: Bring-Up And Driver Installation

    BlueField SW ships with the NVIDIA BlueField Controller Cards. BlueField SW is a reference Linux distribution based on the Yocto Poky distribution and extended to include the NVIDIA OFED stack for Arm and a Linux kernel which supports NVMe-oF. This SW distribution is capable of running all customer- based Linux applications seamlessly.

  • Page 58: Software On Emmc

     Before installing the preferred OS on the BlueField Controller Card, make sure you install the card only in a JBOF system. Installing it in a host system may damage the card. Software On eMMC The BlueField Controller Card boots off eMMC upon power-up. The image flashed on the eMMC from the factory is the Yocto Linux. Run the following command to discover the BlueField Software version: cat /etc/bluefield_version Yocto Distribution Installation...

  • Page 59: Pxe Server Configuration On Host Side

    shutdown -r now Verify the version via: cat /etc/bluefield_version PXE Server Configuration on Host Side Before installing CentOS 7 on the BlueField Controller Card, you need to configure the PXE server on the host side (x86) to allow the deployment of the CentOS image over the BlueField Controller Card.

  • Page 60: Software Requirements

     Please note that CentOS 7.5 is not supported. • Access to the latest BlueField Controller Card SW bundle: NVIDIA uses box.com to distribute BlueField software. Contact your sales/support representative for a custom link to download BlueField software releases. •...

  • Page 61: Setup Procedure With Installation Script

    The “-i” flag points to the OS installation disk. This is the image that is accessed via PXE boot to install the OS on the BlueField Controller Card. • The “-o” flag points to the NVIDIA OFED driver disk for Arm. Download and extract it from  http://www.mellanox.com/page/products_dyn? product_family=34.
  • Page 62 To install the kernel modules, please follow the instruction in section RShim Host Driver. To install the kernel modules, please follow the instruction in NVIDIA BlueField DPU Family Software Documentation under BlueField Software Overview > Installation and Initialization > RShim Host Driver. Step 3: Configure TFTP Server The host should be configured to act as a TFTP server to the BlueField Controller Card via the USB RShim network.
  • Page 63 Patch the initrd with the eMMC driver and TMFIFO (RShim network) driver: mkdir -p /tmp/.bfcentos mkdir -p $/tmp/.bfinstdd cd /tmp/.bfcentos xzcat /var/lib/tftpboot/centos/7.4/initrd-orig.img | cpio -idm mount /root/BlueField-1.0.alpha3.10409/distro/rhel/bluefield_dd/bluefield_dd-4.11.0- 22.el7a.aarch64.iso /tmp/.bfinstddmkdir -p usr/lib/modules/4.11.0-22.el7a.aarch64/updates/cp /tmp/.bfinstdd/lib/modules/4.11.0-22.el7a.aarch64/updates/dw_mmc*.ko usr/lib/mod- ules/ 4.11.0-22.el7a.aarch64/updates/cp /tmp/.bfinstdd/lib/modules/4.11.0-22.el7a.aarch64/updates/tmfifo.ko usr/lib/mod- ules/4.11.0-22.el7a.aarch64/ updates/cp /root/BlueField-1.0.alpha3.10409/distro/rhel/bluefield_dd/bluefield_dd-4.11.0- 22.el7a.aarch64.iso ./ bluefield_dd.iso umount /tmp/.bfinstdd;...
  • Page 64 systemctl restart tftp  Based on the system, the user may need to use “system TFTP restart” instead. Also, if required, the user might need to switch use “xinetd” instead of “TFTP”. Step 4: Set Up the DHCP Server DHCP server set up on the host is required for BlueField Controller Card to get a private IP from the host for PXE boot process completion. Configure the correct server names and domain names so that the BlueField Controller Card can connect to the network via the host later on.

  • Page 65: Flashing Bluefield Controller Card Bootloader Code

    # Specify the IP address for this client. host pxe_client { hardware ethernet 00:1a:ca:ff:ff:01; fixed-address 192.168.100.2; } EOF  It is recommended to back up the previous dhcpd.conf file before overwriting it. Step 5: Set Up the HTTP Server The TFTP server allows the PXE boot to load the initrd and kernel. The BlueField Controller Card obtains all the other required sources through the network, thus, making it necessary to set up an HTTP.

  • Page 66: Opening Terminal Connection To Bluefield Controller Card

    Opening Terminal Connection to BlueField Controller Card To open a console window to the BlueField Controller Card, a terminal application is required. The application “minicom” is used for the flow, however, any standard terminal application can work, e.g. “screen”.  Install minicom by running “yum install minicom”...
  • Page 67 [root@bu-lab02 ~]# ls /dev/rshim0/ boot console net rshim The boot device is used to push the bootstream to the BlueField Controller Card. Upon writing to it, it automatically resets the Arm cores so that it is booted using the pushed bootstream. ...

  • Page 68: Installing Centos 7.4 On Bluefield Controller Card

    Installing CentOS 7.4 on BlueField Controller Card  If the error “no root is found” appears in the installation process, check or disable the firewall as needed on the server host machine. Full PXE Boot Installation Get to the UEFI boot menu. Reboot the BlueField Controller Card by typing “reboot”...
  • Page 69 Navigate to the Boot Manager. UEFI Boot Manager  Select EFI Network, it will then use the TFTP service on the host to discover all available PXE boot options. Shortly after, a “..Fetching Netboot Image” message will appear enabling CentOS installation.  Option to Install CentOS Select CentOS download.
  • Page 70  These configuration inputs are not needed when the kickstart option “-k” is specified when running the setup.sh script. ====================================================================================== ====================================================================================== VNC   Text mode provides a limited set of installation options. It does not offer custom partitioning for full control over the disk layout.
  • Page 71 Europe Pacific 10) Arctic Asia Australia 11) US America 8) Atlantic 12) Etc Africa Indian Antarctica   Please select the timezone. Use numbers or type names directly [b to region list, q to quit]: 11 ====================================================================================== ====================================================================================== Timezone settings   Available timezones in region US Alaska 4) Eastern...
  • Page 72 [ ] File and Print Server 9) [ ] Development and Creative [ ] Basic Web Server Workstation Please make your choice from above ['q' to quit | 'c' to continue | 'r' to refresh]: 9 ====================================================================================== ====================================================================================== Base environment Software selection  ...
  • Page 73 [ ] 3) Use Free Space   Installation requires partitioning of your hard drive. Select what space to use for the install target.   Please make your choice from above ['q' to quit | 'c' to continue | 'r' to refresh]: c ====================================================================================== ====================================================================================== Partition Scheme Options [ ] 1) Standard Partition [ ] 2) Btrfs...
  • Page 74 ====================================================================================== ====================================================================================== Please select new root password. You will have to type it twice.   Password: Password (confirm): ====================================================================================== ====================================================================================== Question   The password you have provided is weak: The password fails the dictionary check it is based on a dictionary word. Would you like to use it anyway?  ...

  • Page 75: Non-Pxe Boot Installation

    Non-PXE Boot Installation When the setup script is run with the “-t” option, it generates a nonpxe.bfb file at the directory where the script is run. The directory contains the install kernel and rootfs which are usually loaded by UEFI during the initial PXE boot stage. Thus, if pushing this file, the host TFTP server no longer needs to be used and UEFI would automatically load the install kernel and rootfs from the boot FIFO.

  • Page 76: Installing Mlnx_Ofed On Arm Cores

    Installing MLNX_OFED on Arm Cores Prerequisite Packages for Installing MLNX_OFED • MLNX_OFED installation requires some prerequisite packages to be installed on the system. Currently, CentOS installed on the BlueField Controller Card has a private network to the host via the USB connection, and it can be used to Secure Copy Protocol (SCP) all the required packages.

  • Page 77: Removing Pre-Installed Kernel Module

    yum install rpm-build yum group install "Development Tools" yum install kernel-devel-`uname -r` yum install valgrind-devel libnl3-devel python-devel yum install tcl tk Note that this is not needed if you installed CentOS 7 with the kickstart (“-k”) option. /auto/sw_mc_soc_project/distro/rhel/kernel-devel-4.11.0-22.el7a.aarch64.rpm Removing Pre-installed Kernel Module There are cases where the kernel is shipped with an earlier version of the mlx5_core driver taken from the upstream Linux code.
  • Page 78 mount /root/MLNX_OFED_LINUX-4.2-1.4.8.0-rhel7.4alternate-aarch64.iso /mnt Install MLNX_OFED. If the kernel on the BlueField is 4.11.0-22.el7a.aarch64, run: cd /mnt # ./mlnxofedinstall --bluefield If the kernel is different than 4.11.0-22.el7a.aarch64, run: cd /mnt # ./mlnxofedinstall --add-kernel-support --skip-repo  For OFED to support DPDK, use the arguments “--upstream-libs” and “--dpdk”. ./mlnxofedinstall --distro rhel7.4alternate --add-kernel-support --upstream-libs --dpdk This step might take longer than expected to be completed.

  • Page 79: Updating Bluefield Controller Card Firmware

    Updating BlueField Controller Card Firmware  The below commands apply to MBF1M616A-CSNAT. The commands vary per OPN. The below steps demonstrate how to manually update the firmware if the automatic process fails. The firmware image can be found in the BlueField Software package.
  • Page 80 To burn the firmware which comes with OFED after OFED is installed, run: /opt/mellanox/mlnx-fw-updater/firmware/mlxfwmanager_sriov_dis -force Power cycle the BlueField Controller Card for the new firmware to take effect. root@bluefield:~# flint -i /opt/fw-Bluefield-red-18_99_4608_MBF1M6X6A-CSNA_Ax- Flexboot-3.5.404_UEFI-14.15.20.bin -d /dev/mst/mt41682_pciconf0 b   Current FW version flash: 18.24.0013 New FW version: 18.99.4608...

  • Page 81: Troubleshooting

    Troubleshooting General Troubleshooting • Reseat the BF2500 Controller Card in its slot or a different slot, if necessary The BF2500 Controller Card no longer works • Try using another cable • Reinstall the drivers for the network driver files may be damaged or deleted •...

  • Page 82: Specifications

    Specifications  Ensure your system can support 75W or greater system power supply through the PCIe x16 interface and an additional 75W through the supplementary 6-pin ATX power supply connector MBF2H516B-EE[N/E]OT Specifications        Form Factor: FHHL  Physical Dimensions: 167.65mm x 11.15mm (6.6in. x 4.37in.) Connector: Dual Ethernet QSFP56 (copper and optical) InfiniBand: IBTA v1.3  ...

  • Page 83: Mbf2H515B-He[N/E]Ot Specifications

    Humidity Operational 10% to 85% relative humidity  Non-operational 10% to 90% relative humidity  Safety: CB / cTUVus / CE Regulatory EMC: CE / FCC / VCCI / ICES / RCM RoHS: RoHS Compliant a. The non-operational storage temperature specifications apply to the product without its package. MBF2H515B-HE[N/E]OT Specifications ...

  • Page 84: Bluefield-2 Bf2500 Mechanical Drawing And Dimensions

    EMC: CE / FCC / VCCI / ICES / RCM RoHS: RoHS Compliant a. The non-operational storage temperature specifications apply to the product without its package BlueField-2 BF2500 Mechanical Drawing and Dimensions  All dimensions are in millimeters. The PCB mechanical tolerance is +/- 0.13mm.

  • Page 85: Bracket Mechanical Drawing And Dimensions

    Bracket Mechanical Drawing and Dimensions...

  • Page 86: Finding The Guid/Mac And Serial Number On The Card

    Finding the GUID/MAC and Serial Number on the Card Each DPU controller d has a different identifier printed on the label: serial number and the card MAC for the Ethernet protocol and the card GUID for the InfiniBand protocol. VPI cards have both a GUID and a MAC (derived from the GUID). ...

  • Page 87: Document Revision History

    Document Revision History Date Comments/Changes May. 2023 Updated Specifications - added non-operational storage temperature specifications Sep. 2021 Updated "System Requirements Overview".  Dec. 2020 • Updated the NCSI interface pinouts. • Removed the UART interface from "Supported Interfaces". • Updated "Package Contents". Dec. 2020 Updated the NCSI interface pinouts.
  • Page 88 NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.
  • Page 89 INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF ANY USE OF THIS DOCUMENT, EVEN IF NVIDIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s aggregate and cumulative liability towards customer for the products described herein shall be limited in...

This manual is also suitable for:

900-9d205-0056-st0900-9d205-0066-st0900-9d205-0058-st0900-9d205-0068-st0Mbf2h516b-eenotMbf2h516b-eeeot ... Show all

Table of Contents