Nvidia BlueField-3 User Manual
  1. Manuals
  2. Brands
  3. Nvidia Manuals
  4. Computer Hardware
  5. BlueField-3
  6. User manual
Nvidia BlueField-3 User Manual

Nvidia BlueField-3 User Manual

Hide thumbs Also See for BlueField-3:
Table of Contents

Advertisement

Quick Links

Download this manual
NVIDIA BlueField-3 DPU User Guide
| 1

Advertisement

Table of Contents
loading

Related Manuals for Nvidia BlueField-3

Summary of Contents for Nvidia BlueField-3

  • Page 1 NVIDIA BlueField-3 DPU User Guide...

  • Page 2: Table Of Contents

    DDR5 SDRAM On-Board Memory ....................29 3.3.6 NC-SI Management Interface ......................29 3.3.7 UART Interface Connectivity ......................30 3.3.8 USB 4-pin RA Connector ........................ 30 3.3.9 1GbE OOB Management Interface ....................31 3.3.9.1 1GbE OOB Management LEDs Interface .................. 31 NVIDIA BlueField-3 DPU User Guide...
  • Page 3 Safety Precautions ..........................51 Unpacking ............................... 51 Pre-Installation Checklist ........................51 Installation Instructions .......................... 52 Cables and Modules ..........................52 5.8.1 Networking Cable Installation ....................... 52 DPU Power-Up and Power-Down Sequences ..................54 5.9.1 Power-Up Sequence ........................54 NVIDIA BlueField-3 DPU User Guide...
  • Page 4 Finding the GUID/MAC on the DPU ..................83 PCIe Auxiliary Card Kit ......................85 11.1 PCIe Auxiliary Card Package Contents ....................86 11.2 Channel Insertion Loss ..........................87 11.3 Cabline CA-II Plus Harness Pinouts ......................87 NVIDIA BlueField-3 DPU User Guide...
  • Page 5 PCIe Auxiliary Card Mechanical Drawings and Dimensions ............99 11.4.2 Bracket Mechanical Drawings and Dimensions ................99 11.4.3 Cabline CA-II Plus Harnesses Mechanical Drawing ..............100 Supported Servers and Power Cords ..................101 Document Revision History ....................102 NVIDIA BlueField-3 DPU User Guide...
  • Page 6 This User Manual describes NVIDIA® BlueField®-3 InfiniBand/Ethernet DPU (Data Processing Unit). It provides details as to the interfaces of the board, specifications, required software and firmware for operating the board, and a step-by-step plan of how to bring up the BlueField-3 DPUs.
  • Page 7 Gen 5.0 Production FHHL Arm- (Default) Cores Ethernet: 400GbE ✔ ✔ ✔ 900-9D3B6-00SN-AB0 B3240 P-Series InfiniBand: 2-Ports PCIe 32GB Tall 41692 MT_0000000964 Mass / 16 NDR 400Gb/s QSFP112 Gen 5.0 Production Arm- (Default) Cores Ethernet: 400GbE NVIDIA BlueField-3 DPU User Guide...
  • Page 8 This manual is intended for the installer and user of these cards. The manual assumes basic familiarity with InfiniBand/Ethernet network and architecture specifications. Technical Support Customers who purchased NVIDIA products directly from NVIDIA are invited to contact us through the following methods: • URL: www.nvidia.com...
  • Page 9 Active Optical Cables (AOCs) and transceivers in a wide range of lengths from 0.5m to 10km. In addition to meeting IBTA standards, NVIDIA tests every product in an end-to-end environment ensuring a Bit Error Rate of less than 1E-15.

  • Page 10: Introduction

    Providing powerful computing, and a broad range of programmable acceleration engines in the I/O path, BlueField-3 is perfectly positioned to address the infrastructure needs of the most demanding applications, while delivering full software backward compatibility through the NVIDIA DOCA™...

  • Page 11: Package Contents

    Description Operating BlueField-3 DPU is shipped with Ubuntu – a Linux commercial operating system – which includes the NVIDIA OFED stack (MLNX_OFED), and is System capable of running all customer-based Linux applications seamlessly. For more information, please refer to the DOCA SDK documentation or NVIDIA BlueField-3 Software User Manual.

  • Page 12: Accessories Kit

    This is an optional kit which applies to following OPNs: 900-9D3B6-00CV-AA0, 900-9D3B6-00SV-AA0, 900-9D3B6-00CC-AA0, 900-9D3B6-00SC-AA0, 900- 9D3B6-00CN-AB0 and 900-9D3B6-00SN-AB0. The PCIe auxiliary kit can be purchased separately to operate selected DPUs in a dual-socket server. For package contents, refer to PCIe Auxiliary Card Kit. NVIDIA BlueField-3 DPU User Guide | 12...

  • Page 13: Features And Benefits

    This section describes hardware features and capabilities. Please refer to the relevant driver and/or firmware release notes for feature availability. Feature Description InfiniBand Architecture BlueField-3 DPU delivers low latency, high bandwidth, and computing efficiency for high-performance computing (HPC), Specification v1.5 compliant artificial intelligence (AI), and hyperscale cloud data centers applications. BlueField-3 DPU is InfiniBand Architecture Specification v1.5 compliant.
  • Page 14 40GB with 30K Write Cycles eMMC for SoC BIOS and OS. • SSD (onboard BGA) 128GByte for user SoC OS, logs and application SW. • DDR5 SDRAM - 16GB/32GB @5600MT/s single/dual-channel DDR5 SDRAM memory. Solder down on-board. 128bit + 16bit ECC NVIDIA BlueField-3 DPU User Guide | 14...
  • Page 15 Description BlueField-3 IC The NVIDIA BlueField-3 DPU integrates x8 / x16 Armv8.2+ A78 Hercules cores (64-bit) is interconnected by a coherent mesh network, one DRAM controller, an RDMA intelligent network adapter supporting up to 400Gb/s, an embedded PCIe switch with endpoint and root complex functionality, and up to 32 lanes of PCIe Gen 5.0.
  • Page 16 BlueField-3 DPU functions as a “computer-in-front-of-a-computer,” unlocking unlimited opportunities for custom security applications on its Arm processors, fully isolated from the host’s CPU. In the event of a compromised host, BlueField-3 may detect/block malicious activities in real-time and at wire speed to prevent the attack from spreading further.

  • Page 17: Bluefield Dpu Administrator Quick Start Guide

    BlueField DPU using the DOCA framework, please refer to the NVIDIA DOCA Developer Quick Start Guide. Not sure which guide to follow? For more details on the different BlueField user types, please refer to the NVIDIA BlueField and DOCA User Types document.

  • Page 18: Connecting To Bluefield And Verifying Version

    17:00.0 Ethernet controller: Mellanox Technologies MT43244 BlueField-3 integrated ConnectX-7 network controller (rev 01) 17:00.1 Ethernet controller: Mellanox Technologies MT43244 BlueField-3 integrated ConnectX-7 network controller (rev 01) 17:00.2 DMA controller: Mellanox Technologies MT43244 BlueField-3 SoC Management Interface (rev 01) 3. If an older DOCA software version is installed on your host, make sure to uninstall it before proceeding with the installation of the new version:...

  • Page 19: Updating Bluefield Bfb Image

    Collecting BlueField booting status. Press Ctrl+C to stop… INFO[BL2]: start … INFO[MISC]: Ubuntu installation started INFO[MISC]: Installation finished INFO[MISC]: Rebooting... 2. Upgrade the firmware of the BlueField DPU: Upgrade the BlueField DPU's firmware: NVIDIA BlueField-3 DPU User Guide | 19...
  • Page 20 3. Verify that the BFB has been installed and the firmware has been upgraded successfully by accessing the DPU again: SSH to the BlueField DPU from the host using OOB IP or RShim IP: ssh <ip> b. Check the versions of the DPU image and firmware: # sudo bfvcheck Expected output example: NVIDIA BlueField-3 DPU User Guide | 20...

  • Page 21: Additional Reading

    ATF: v2.2(release):4.0.3-3-g886241c UEFI: 4.0.3-1-g2162ecf FW: 32.37.1300 Version check complete. No issues found. 2.4 Additional Reading To learn more about BlueField please see: • BlueField hardware troubleshooting • BlueField software installation • BlueField software troubleshooting NVIDIA BlueField-3 DPU User Guide | 21...

  • Page 22: Supported Interfaces

    The below figures are for illustration purposes only and might not reflect the current revision of the DPU. 3.1 HHHL DPU Layout and Interface Information DPU Component Side DPU Print Side HHHL Single-Slot DPU Model: B3140H 900-9D3D4-00EN-HA0 900-9D3D4-00NN-HA0 NVIDIA BlueField-3 DPU User Guide | 22...
  • Page 23 USB 4-pin RA Connector Used for OS image loading 1GbE OOB Management 1GbE BASE-T OOB management interface Interface Integrated BMC DPU BMC SSD Interface 128GB RTC Battery Battery holder for RTC eMMC x8 NAND flash NVIDIA BlueField-3 DPU User Guide | 23...

  • Page 24: Fhhl Dpu Layout And Interface Information

    3.2 FHHL DPU Layout and Interface Information DPU Component Side DPU Print Side FHHL Single-Slot Dual-Port DPUs with PCIe Extension Option Model: B3220 and B3210 900-9D3B6-00CV-AA0 900-9D3B6-00SV-AA0 900-9D3B6-00CC-EA0 900-9D3B6-00SC-EA0 900-9D3B6-00CC-AA0 900-9D3B6-00SC-AA0 FHHL Single-Slot Single-Port DPUs Model: B3140L 900-9D3B4-00EN-EA0 900-9D3B4-00PN-EA0 NVIDIA BlueField-3 DPU User Guide | 24...
  • Page 25 DPU Component Side DPU Print Side FHHL Dual-Slot Dual-Port DPUs Model: B3240 900-9D3B6-00CN-AB0 900-9D3B6-00SN-AB0 NVIDIA BlueField-3 DPU User Guide | 25...
  • Page 26 Item Interface Description • DPU System on Chip BlueField-3 P-Series - 16 Arm-Cores - 560MHz/2133MHz • BlueField-3 E-Series - 8 Arm-Cores - 505MHz/2000MHz Networking Interface The network traffic is transmitted through the DPU QSFP112 connectors. The QSFP112 connectors allow the use of...

  • Page 27: Interfaces Detailed Description

    Leverages the rich Arm software ecosystem and introduces the ability to offload the x86 software stack. At the heart of BlueField-3, the ConnectX-7 network offload controller with RDMA and RDMA over Converged Ethernet (RoCE) technology delivers cutting-edge performance for networking and storage applications such as NVMe over Fabrics.

  • Page 28: Networking Ports Leds Interface

    C access to the networking ports fails Blinks until error is fixed Over-current Over-current condition of the networking ports Blinks until error is fixed Physical Activity Blinking Green Link Up Solid Green Physical Up (InfiniBand Mode Only) Solid Yellow NVIDIA BlueField-3 DPU User Guide | 28...

  • Page 29: Pci Express Interface

    The DPU enables the connection of a Baseboard Management Controller (BMC) to a set of Network Interface Controller (NICs) to enable out-of-band remote manageability. The NC-SI management is supported over RMII and has a connector on the DPU. Please refer to NC-SI Management Interface for pins. NVIDIA BlueField-3 DPU User Guide | 29...

  • Page 30: Uart Interface Connectivity

    A UART debug interface is available on DPU cards via a 20-pin NC-SI connector. For DPUs without onboard BMC hardware, the UART interface is that of the BlueField-3 device. For DPUs with onboard BMC hardware, the UART interface is that of the NIC BMC device. The connectivity for both cases is shown in the...

  • Page 31: 1Gbe Oob Management Interface

    The DPU PTP solution allows you to run any PTP stack on your host. With respect to testing and measurements, selected NVIDIA DPUs allow you to use the PPS-out signal from the onboard MMCX RA connecter. The DPU also allows measuring PTP in scale with the PPS-In signal. The PTP HW clock on the Network adapter is sampled on each PPS-In signal, and the timestamp is sent to the SW.

  • Page 32: External Pcie Power Supply Connector

    • Do not link the CPU power cable to the BlueField-3 DPU's PCIe Aux power, as their pin configurations differ. Utilizing the CPU power cable in this manner is strictly prohibited and can potentially damage the BlueField-3 DPU. Please refer to...

  • Page 33: Cabline Ca-Ii Plus Connectors

    The DPU incorporates an onboard integrated NIC BMC and an Ethernet switch. The BMC becomes available once the host server powers up the card. The NIC BMC can control the DPU's power and enables DPU shutdown and power-up. NVIDIA BlueField-3 DPU User Guide | 33...

  • Page 34: Nvme Ssd Interface

    The DPU incorporates a Coin type Lithium battery CR621 for RTC (Real Time Clock). 3.3.16 eMMC Interface The eMMC is an x8 NAND flash used for Arm boot and operating system storage. Memory size is 128GB, where it is effectively pSLC 40GB. NVIDIA BlueField-3 DPU User Guide | 34...

  • Page 35: Pinouts Description

    EA0 & 900-9D3B4-00SC-EA0 only) A11 PERST# PCIe Reset B11 WAKW#/RSVD A12 GND B12 RSVD A13 REFCLK+ Host Reference Clock B13 GND A14 REFCLK- Host Reference Clock B14 PETP0 A15 GND B15 PETN0 A16 PERP0 B16 GND NVIDIA BlueField-3 DPU User Guide | 35...
  • Page 36 B34 PETN4 A35 PERP4 B35 GND A36 PERN4 B36 GND A37 GND B37 PETP5 A38 GND B38 PETN5 A39 PERP5 B39 GND A40 PERN5 B40 GND A41 GND B41 PETP6 A42 GND B42 PETN6 NVIDIA BlueField-3 DPU User Guide | 36...
  • Page 37 B60 GND A61 PERN10 B61 GND A62 GND B62 PETP11 A63 GND B63 PETN11 A64 PERP11 B64 GND A65 PERN11 B65 GND A66 GND B66 PETP12 A67 GND B67 PETN12 A68 PERP12 B68 GND NVIDIA BlueField-3 DPU User Guide | 37...
  • Page 38 A75 GND B75 PETN14 A76 PERP14 B76 GND A77 PERN14 B77 GND A78 GND B78 PETP15 A79 GND B79 PETN15 A80 PERP15 B80 GND A81 PERN15 B81 PRSNT2# Mechanical Present A82 GND B82 GND NVIDIA BlueField-3 DPU User Guide | 38...

  • Page 39: External Power Supply Connector

    The mechanical pinout of the 8-pin external +12V power connector is shown below. The +12V connector is a GPU power PCIe standard connector. Care should be taken to ensure the power is applied to the correct pins as some 8-pin ATX-type connectors can have different pinouts. Pin Number Description Sense1 Sense0 NVIDIA BlueField-3 DPU User Guide | 39...

  • Page 40: Nc-Si Management Interface

    For DPUs, this pin should be connected between the connector and the RBT PHY. External termination determined by the DPU RBT PHY requirements. Ground RBT_CRS_DV Output Carrier sense/receive data valid. This signal is used to indicate to the baseboard that the carrier sense/receive data is valid. NVIDIA BlueField-3 DPU User Guide | 40...
  • Page 41 RBT signals are isolated. For DPUs, this pin should be connected between the connector and the RBT PHY. External termination determined by the DPU RBT PHY requirements. UART_TX Input 3.3V UART TX signal from the baseboard NVIDIA BlueField-3 DPU User Guide | 41...
  • Page 42 RBT_ARB_OUT pin of the next populated card and so on sequentially for all cards on the specified RBT bus to ensure the arbitration ring is complete. This logic should bypass slots that are not populated or powered off. NVIDIA BlueField-3 DPU User Guide | 42...

  • Page 43: Cabline Ca-Ii Plus Connectors Pinouts

    Micro coax PCIE_CPU_CX_13P Micro coax GND BAR PCIE_CPU_CX_12N Micro coax PCIE_CPU_CX_12P Micro coax GND BAR PCIE_CPU_CX_11N Micro coax PCIE_CPU_CX_11P Micro coax GND BAR PCIE_CPU_CX_10N Micro coax PCIE_CPU_CX_10P Micro coax GND BAR PCIE_CPU_CX_9N Micro coax NVIDIA BlueField-3 DPU User Guide | 43...
  • Page 44 Micro coax PCIE_CPU_CX_4P Micro coax GND BAR PCIE_CPU_CX_3N Micro coax PCIE_CPU_CX_3P Micro coax GND BAR PCIE_CPU_CX_2N Micro coax PCIE_CPU_CX_2P Micro coax GND BAR PCIE_CPU_CX_1N Micro coax PCIE_CPU_CX_1P Micro coax GND BAR PCIE_CPU_CX_0N Micro coax NVIDIA BlueField-3 DPU User Guide | 44...
  • Page 45 Pin# on other end PCIE_CPU_CX_0P Micro coax GND BAR I2C_DPU_BMC_SDA Micro coax I2C_DPU_BMC_SCL Micro coax AUX_PGOOD Micro coax No wire Micro coax I2C_AUX_SCL Micro coax I2C_AUX_SDA Micro coax S_PRSNT1_L Micro coax No wire NVIDIA BlueField-3 DPU User Guide | 45...

  • Page 46: Print Side

    Micro coax PCIE_CPU_CX_1N Micro coax GND BAR PCIE_CPU_CX_2P Micro coax PCIE_CPU_CX_2N Micro coax GND BAR PCIE_CPU_CX_3P Micro coax PCIE_CPU_CX_3N Micro coax GND BAR PCIE_CPU_CX_4P Micro coax PCIE_CPU_CX_4N Micro coax GND BAR PCIE_CPU_CX_5P Micro coax NVIDIA BlueField-3 DPU User Guide | 46...
  • Page 47 Micro coax PCIE_CPU_CX_10N Micro coax GND BAR PCIE_CPU_CX_11P Micro coax PCIE_CPU_CX_11N Micro coax GND BAR PCIE_CPU_CX_12P Micro coax PCIE_CPU_CX_12N Micro coax GND BAR PCIE_CPU_CX_13P Micro coax PCIE_CPU_CX_13N Micro coax GND BAR PCIE_CPU_CX_14P Micro coax NVIDIA BlueField-3 DPU User Guide | 47...
  • Page 48 Pin# Signal Name Wire Type AWG# Pin# on other end PCIE_CPU_CX_14N Micro coax GND BAR PCIE_CPU_CX_15P Micro coax PCIE_CPU_CX_15N Micro coax GND BAR S_PERST1_CONN_L Micro coax No wire No Wire S_PRSNT2_L Micro coax NVIDIA BlueField-3 DPU User Guide | 48...

  • Page 49: Hardware Installation

    Install the DPU according to the form-factor you have purchased. Refer to DPU Installation Connect cables or modules to the DPU Refer to Cables and Modules Installation Power-up the DPU Refer to DPU Power-Up Instructions NVIDIA BlueField-3 DPU User Guide | 49...

  • Page 50: System Requirements

    Introduction section. • Software Stacks - The DPU is shipped with Linux based Operating System burned on it which includes all needed drivers. For more information, please refer to the Software User Manual. NVIDIA BlueField-3 DPU User Guide | 50...

  • Page 51: Safety Precautions

    Turn off the power to the system, and disconnect the power cord. Refer to the system documentation for instructions. Before you install the DPU, make sure that the system is disconnected from power. NVIDIA BlueField-3 DPU User Guide | 51...

  • Page 52: Installation Instructions

    Insert the connector into the DPU. Be careful to insert the connector straight into the cage. Do not apply any torque, up or down, to the connector cage in the DPU. d. Make sure that the connector locks in place. When installing cables make sure that the latches engage. NVIDIA BlueField-3 DPU User Guide | 52...
  • Page 53 6. To remove a cable, disengage the locks and slowly pull the connector away from the port receptacle. LED indicator will turn off when the cable is unseated. NVIDIA BlueField-3 DPU User Guide | 53...

  • Page 54: Dpu Power-Up And Power-Down Sequences

    1. The 12V_ATX voltage can exceed the 12V_PCIE voltage by a maximum allowable voltage difference (dV) when the 12V_ATX reaches 11V. See below graph and table describing the dV between the 12V_ATX and 12V_PCIE voltages. NVIDIA BlueField-3 DPU User Guide | 54...

  • Page 55: Power-Down Sequence

    1. The 12V_PCIE voltage can be powered down simultaneously with the 12V_ATX voltage, or within a maximum delay of 5 milliseconds. The below graph illustrates the delay between the 12_ATX and 12_PCIE voltages at power-down. 2. The 12V_PCIE voltage must not be powered down while the 12V_ATX voltage is powered up. NVIDIA BlueField-3 DPU User Guide | 55...

  • Page 56: Pcie X16 Dpus Installation Instructions

    To power-up the FHHL P-Series DPUs, you need to connect a PCIe external power cable. The PCIe external power cable should be supplied by the customer. Refer to External Power Supply Connector for pin descriptions. 1. Open the system case. 2. Place the DPU in an available PCI Express slot. NVIDIA BlueField-3 DPU User Guide | 56...
  • Page 57 4. When the DPU is properly seated, the port connectors are aligned with the slot opening, and the DPU faceplate is visible against the system chassis. 5. Secure the DPU with the screw. NVIDIA BlueField-3 DPU User Guide | 57...

  • Page 58: Uninstalling The Dpu

    4. Holding the DPU from its center, gently pull the DPU out of the PCI Express slot. 5. When the port connectors reach the top of the chassis window, gently pull the DPU in parallel to the motherboard. NVIDIA BlueField-3 DPU User Guide | 58...

  • Page 59: Pcie Extension Option (2X Pcie X16) Installation Instructions

    The Cabline harness marked with a "WHITE" label should be connected to the connector on the DPU and Auxiliary PCIe card engraved with “White Cable”, while the one marked with a "BLACK" label should be connected to the connector on the DPU and Auxiliary PCIe card engraved with “Black Cable”. The harnesses' minimal bending radius is 10[mm]. NVIDIA BlueField-3 DPU User Guide | 59...

  • Page 60: Installing The Dpu

    Step 2: Plug the Cabline CA-II Plus harnesses on the DPU while paying attention to the color-coding. As indicated on both sides of the card; plug the black harness to the component side and the white harness to the print side. NVIDIA BlueField-3 DPU User Guide | 60...
  • Page 61 Step 4: Slide the retention clip latches through the cutouts on the PCB. The latches should face the annotation on the PCB. Step 4: Clamp the retention clip. Verify both latches are firmly locked. NVIDIA BlueField-3 DPU User Guide | 61...
  • Page 62 Step 6: Plug the Cabline CA-II Plus harnesses on the PCIe Auxiliary Card. As indicated on both sides of the Auxiliary connection card; plug the black harness to the component side and the white harness to the print side. NVIDIA BlueField-3 DPU User Guide | 62...
  • Page 63 Step 8: Slide the retention clip through the cutouts on the PCB. Make sure latches are facing "Black Cable" annotation as seen in the below picture. Step 9: Clamp the retention clip. Verify both latches are firmly locked. NVIDIA BlueField-3 DPU User Guide | 63...

  • Page 64: Uninstalling The Cards

    3. To remove the card, disengage the retention mechanisms on the brackets (clips or screws). 4. Holding the DPU from its center, gently pull the DPU and Auxiliary Connections card out of the PCI Express slot. NVIDIA BlueField-3 DPU User Guide | 64...

  • Page 65: Setting High-Speed-Port Link Type

    6 Setting High-Speed-Port Link Type The following table lists the BlueField-3 supported speeds and the default networking port link type per OPN. Model Data Transmission Rate Default Protocol and Rate B3140L 900-9D3B4-00EN-EA0 / 900-9D3B4-00PN-EA0 InfiniBand: NDR 400Gb/s InfiniBand NDR 400Gb/s...

  • Page 66: Troubleshooting

    Check that both the DPU and its link are set to the same speed and duplex settings Forgot password needed to Refer to the latest version of BlueField DPU SW Manual and follow install/upgrade the DPU image instructions under "Upgrading NVIDIA BlueField DPU Software" section. NVIDIA BlueField-3 DPU User Guide | 66...

  • Page 67: Specifications

    75W system power supply through the PCIe x16 interface. P-Series DPUs: A minimum of 75W or greater system power supply through the PCIe x16 interface, and a supplementary 8-pin ATX power supply connectivity available through the external power supply connector NVIDIA BlueField-3 DPU User Guide | 67...

  • Page 68: B3210 Dpus Specifications

    C2M, XLAUI C2M , XLPPI, SFI Electrical and Thermal Voltage: 12V Specifications Electrical and thermal specifications are provided in "NVIDIA BlueField-3 DPUs Electrical and Thermal Specifications" document. You can access the document either by logging into NVOnline or by contacting your NVIDIA representative. Environmental Temperature Operational 0°C to 55°C...
  • Page 69 RoHS compliant Notes: The BlueField-3 DPU supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA InfiniBand product. The non-operational storage temperature specifications apply to the product without its package.

  • Page 70: B3220 Dpus Specifications

    10GBASE-CX4, 1000BASE-CX, CAUI-4 C2M, 25GAUI C2M, XLAUI C2M , XLPPI, SFI Electrical and Voltage: 12V Thermal Electrical and thermal specifications are provided in "NVIDIA BlueField-3 DPUs Electrical and Thermal Specifications" document. You Specifications can access the document either by logging into NVOnline or by contacting your NVIDIA representative. Environmental...
  • Page 71 RoHS compliant Notes: The BlueField-3 DPU supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA InfiniBand product. The non-operational storage temperature specifications apply to the product without its package.

  • Page 72: B3240 Dpus Specifications

    PCI Express 5.0: SERDES @ 32.0GT/s, 16 lanes (4.0, 3.0, 2.0 and 1.1 compatible) Electrical and Voltage: 12V Thermal Electrical and thermal specifications are provided in "NVIDIA BlueField-3 DPUs Electrical and Thermal Specifications" document. You can Specifications access the document either by logging into NVOnline or by contacting your NVIDIA representative. Environmental...
  • Page 73 RoHS compliant Notes: The BlueField-3 DPU supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA InfiniBand product. The non-operational storage temperature specifications apply to the product without its package.

  • Page 74: B3140L Dpus Specifications

    25GBASE-R, 10GBASE-R, 10GBASE-CX4, 1000BASE-CX, CAUI-4 C2M, 25GAUI C2M, XLAUI C2M , XLPPI, SFI Electrical and Voltage Thermal Electrical and thermal specifications are provided in "NVIDIA BlueField-3 DPUs Electrical and Thermal Specifications" document. You Specifications can access the document either by logging into NVOnline or by contacting your NVIDIA representative. Environmental...
  • Page 75 RoHS compliant Notes: The BlueField-3 DPU supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA InfiniBand product. The non-operational storage temperature specifications apply to the product without its package.

  • Page 76: B3140H Dpus Specifications

    Specifications Environmental Temperature Operational 0°C to 55°C Non-operational -40°C to 70°C Humidity Operational 10% to 85% relative humidity Non-operational 10% to 90% relative humidity Altitude (Operational) 3050m Safety CB / cTUVus / CE NVIDIA BlueField-3 DPU User Guide | 76...

  • Page 77: Dpus Mechanical Drawing And Dimensions

    RoHS compliant Notes: The BlueField-3 DPU supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA InfiniBand product. The non-operational storage temperature specifications apply to the product without its package.
  • Page 78 OPNs Mechanical Drawings FHHL Single-Slot Single-Port DPUs Model: B3140L 900-9D3B4-00EN-EA0 900-9D3B4-00PN-EA0 FHHL Single-Slot Dual-Port DPUs with PCIe Extension Option Model: B3220 900-9D3B6-00CV-AA0 900-9D3B6-00SV-AA0 Model: B3210 900-9D3B6-00CC-AA0 900-9D3B6-00SC-AA0 NVIDIA BlueField-3 DPU User Guide | 78...
  • Page 79 OPNs Mechanical Drawings FHHL Dual-Slot Dual-Port DPUs Model: B3240 900-9D3B6-00CN-AB0 900-9D3B6-00SN-AB0 NVIDIA BlueField-3 DPU User Guide | 79...

  • Page 80: Bracket Mechanical Drawings

    Model/OPNs Tall Bracket Short Bracket HHHL Single-Slot Model: B3140H Dual-Port DPUs 900-9D3D4-00EN-HA0 900-9D3D4-00NN-HA0 FHHL Single-Slot Model: B3140L Single-port DPUs 900-9D3B4-00EN-EA0 900-9D3B4-00PN-EA0 FHHL Single-Slot Model: B3220 Dual-Port DPUs 900-9D3B6-00CV-AA0 900-9D3B6-00SV-AA0 Model: B3210 900-9D3B6-00CC-AA0 900-9D3B6-00SC-AA0 NVIDIA BlueField-3 DPU User Guide | 80...
  • Page 81 DPU Configuration Model/OPNs Tall Bracket Short Bracket FHHL Dual-Slot Model: B3240 Dual-port DPUs 900-9D3B6-00CN-AB0 900-9D3B6-00SN-AB0 NVIDIA BlueField-3 DPU User Guide | 81...

  • Page 82: Monitoring

    Card Configuration Maximum Dimensions Single-slot DPUs 900-9D3B4-00EN-EA0, 900-9D3B4-00PN-EA0, Length, Width, Height: 139.6mm x 92.7mm x 10.2mm 900-9D3B6-00CV-AA0, 900-9D3B6-00SV-AA0, 900-9D3B6-00CC-AA0, 900-9D3B6-00SC-AA0, 900-9D3B6-00CC-EA0, 900-9D3B6-00SC-EA0 Dual-slot DPUs 900-9D3B6-00CN-AB0, 900-9D3B6-00SN-AB0 Length, Width, Height: 139.6mm x 92.7mm x 29.3mm NVIDIA BlueField-3 DPU User Guide | 82...

  • Page 83: Finding The Guid/Mac On The Dpu

    The allocation of MAC addresses to the embedded CPU is derived from a few configuration factors which set some variables. The gap between the MAC addresses is set by constant numbers in HEX. The OOB, ECPF, and MPF MAC addresses depend on the BASE MAC address. NVIDIA BlueField-3 DPU User Guide | 83...
  • Page 84 DPU Board Label (Example) NVIDIA BlueField-3 DPU User Guide | 84...

  • Page 85: Pcie Auxiliary Card Kit

    (top side) of the network card, while the White Cabline SA-II Plus harnesses mates with the pint side (bottom side) of the main network card. For hardware installation, please refer to PCIe Extension Option (2x PCIe x16) Installation Instructions. NVIDIA BlueField-3 DPU User Guide | 85...

  • Page 86: Pcie Auxiliary Card Package Contents

    Cabline CA-II Plus harness (black) - Length according to kit OPN (15, 25 or 35cm) Accessories Retention Clip for Cabeline harness (shipped assembled on the harnesses - optional) PCIe Auxiliary card short bracket PCIe Auxiliary card tall bracket (shipped assembled on the Auxiliary card) NVIDIA BlueField-3 DPU User Guide | 86...

  • Page 87: Channel Insertion Loss

    The PCI-SIG Gen5 SPEC also defines the total loss for AIC (bump to GF) to be 9.5dB @16Ghz. The BlueField-3 AIC, together with a 15cm Cabline CA-II Plus harnesses and the Passive PCIe Auxiliary Card loss is: 3dB+2dB+3.8dB+1.5dB=10.3dB > 9.5dB 11.3 Cabline CA-II Plus Harness Pinouts...
  • Page 88 Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point PCIE_CPU_CX_12N Micro Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point NVIDIA BlueField-3 DPU User Guide | 88...
  • Page 89 Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point PCIE_CPU_CX_8P Micro Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point NVIDIA BlueField-3 DPU User Guide | 89...
  • Page 90 Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point PCIE_CPU_CX_3P Micro Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point NVIDIA BlueField-3 DPU User Guide | 90...
  • Page 91 Connect from the Black Cabline CA-II Plus cable through 220nF capacitors to the x16 PCIe Tx lanes of the coax CPU/GPU/End-Point I2C_DPU_BMC_SDA Micro coax I2C_DPU_BMC_SCL Micro coax AUX_PGOOD Micro coax No wire Micro coax NVIDIA BlueField-3 DPU User Guide | 91...
  • Page 92 Cabline CA-II Plus interface parameters to the main- card BlueField DPU silicon, like Cabline CA-II Plus cables length (contact NVIDIA for the format of this EEPROM). If additional optional I2C slave devices need to be managed by the main-card BlueField DPU silicon, they need to be included on this I2C bus as well.

  • Page 93: Cabline Ca-Ii Plus Harness - Print Side

    Reserved for future expansion coax PCIE_CX_CPU_0P Micro Connect from the White Cabline CA-II Plus harness to the PCIe x16 Rx lanes of the CPU/GPU/End-Point. coax 220nF caps are required on this signal on the BlueField DPU. NVIDIA BlueField-3 DPU User Guide | 93...
  • Page 94 220nF caps are required on this signal on the BlueField DPU. PCIE_CX_CPU_4N Micro Connect from the White Cabline CA-II Plus harness to the PCIe x16 Rx lanes of the CPU/GPU/End-Point. coax 220nF caps are required on this signal on the BlueField DPU. NVIDIA BlueField-3 DPU User Guide | 94...
  • Page 95 220nF caps are required on this signal on the BlueField DPU. PCIE_CX_CPU_9N Micro Connect from the White Cabline CA-II Plus harness to the PCIe x16 Rx lanes of the CPU/GPU/End-Point. coax 220nF caps are required on this signal on the BlueField DPU. NVIDIA BlueField-3 DPU User Guide | 95...
  • Page 96 220nF caps are required on this signal on the BlueField DPU. PCIE_CX_CPU_14P Micro Connect from the White Cabline CA-II Plus harness to the PCIe x16 Rx lanes of the CPU/GPU/End-Point. coax 220nF caps are required on this signal on the BlueField DPU. NVIDIA BlueField-3 DPU User Guide | 96...
  • Page 97 Cabline harnesses are connected. Read logic 1 (3.3V) if one or both the Cabline harnesses are not connected. No wires are connected to these pins to ensure they do not interfere with the operation of S_PRSNT1_L for the detection when the two Cabline harnesses are installed. NVIDIA BlueField-3 DPU User Guide | 97...

  • Page 98: Technical Specifications

    10% to 85% relative humidity Non-operational 10% to 90% relative humidity Altitude (Operational) 3050m Regulatory Safety: CB / cTUVus / CE EMC: CE / FCC / VCCI / ICES / RCM / KC RoHS: RoHS Compliant NVIDIA BlueField-3 DPU User Guide | 98...

  • Page 99: Pcie Auxiliary Card Mechanical Drawings And Dimensions

    11.4.1 PCIe Auxiliary Card Mechanical Drawings and Dimensions All dimensions are in millimeters. The PCB mechanical tolerance is +/- 0.13mm. 11.4.2 Bracket Mechanical Drawings and Dimensions Auxiliary PCIe Connection Card Tall Bracket Auxiliary PCIe Connection Card Short Bracket NVIDIA BlueField-3 DPU User Guide | 99...

  • Page 100: Cabline Ca-Ii Plus Harnesses Mechanical Drawing

    11.4.3 Cabline CA-II Plus Harnesses Mechanical Drawing NVIDIA BlueField-3 DPU User Guide | 100...

  • Page 101: Supported Servers And Power Cords

    12 Supported Servers and Power Cords To be updated in a future revision of this document. NVIDIA BlueField-3 DPU User Guide | 101...

  • Page 102: Document Revision History

    Updated Specifications - added non-operational storage temperature specifications. • Updated Ethernet protocols in Specifications. Apr 2023 Added PSID and device ID information in NVIDIA BlueField-3 DPU User Guide. Mar 2023 Updated DDR5 SDRAM On-Board Memory. NVIDIA BlueField-3 DPU User Guide | 102...
  • Page 103 BB81 and B882. • Added heatsink dimensions in Introduction and Specifications. Jun 2022 Renamed the document from "NVIDIA BleuField-3 InfiniBand/VPI DPU User Guide" to "NVIDIA BlueField-3 DPU User Guide" May 2022 First release NVIDIA BlueField-3 DPU User Guide | 103...
  • Page 104 No license, either expressed or implied, is granted under any NVIDIA patent right, copyright, or other NVIDIA intellectual property right under this document. Information published by NVIDIA regarding third-party products or services does not constitute a license from NVIDIA to...

Table of Contents