Related Manuals for Nvidia BlueField BF1500
Summary of Contents for Nvidia BlueField BF1500
- Page 1 NVIDIA BlueField BF1500 Ethernet Controller Card User Guide Exported on Oct/31/2023 01:39 PM...
-
Page 2: Table Of Contents
Table of Contents Introduction............................7 Product Overview ..............................7 Features and Benefits ............................8 Operating Systems/Distributions ..........................11 Connectivity ..............................11 Package Contents ...............................11 Supported Interfaces ..........................12 BF1500 Controller Card Layout and Interface Information ....................12 Interfaces Detailed Description..........................14 BlueField System-on-Chip (SoC) .......................... 14 Ethernet QSFP28 Interface .......................... - Page 3 Pin Description ............................19 PCI Express Pins Description...........................19 NC-SI Management Interface ..........................22 JTAG CoreSight 10 ..............................24 Hardware Installation ..........................26 Safety Warnings ..............................26 Installation Procedure Overview..........................26 System Requirements ............................27 Hardware Requirements ........................... 27 Airflow Requirements ..........................27 Software Requirements ............................ 28 Safety Precautions ..............................28 Unpacking the Package ............................29 Pre-Installation Checklist ............................29...
- Page 4 On Linux ..............................35 Uninstalling the BF1500 Controller Card ........................35 Bring-Up and Driver Installation ........................38 Troubleshooting ............................39 General Troubleshooting ............................39 Specifications............................40 MBF1L516B-CSNAT Specifications..........................40 Mechanical Drawing and Dimensions.........................42 Mechanical Drawing of the BF1500 Controller Card - Component Side................42 Mechanical Drawing of the BF1500 Controller Card - Print Side..................
- Page 5 About This Manual This User Manual describes NVIDIA® BlueField® Ethernet BF1500 Controller Card. It provides details as to the interfaces of the board, specifications, required software and firmware for operating the board, and a step-by-step plan of how to bring up the BlueField Controller Card.
- Page 6 0.5m to 10km. In addition to meeting IBTA standards, Mellanox tests every product in an end-to-end environment ensuring a Bit Error Rate of less than 1E-15. Read more at https://www.nvidia.com/en-us/networking/interconnect/ Document Conventions When discussing memory sizes, MB and MBytes are used in this document to mean size in mega Bytes. The use of Mb or Mbits (small b) indicates size in mega bits.
-
Page 7: Introduction
Product Overview The BlueField BF1500 Controller Card is the perfect solution for managing NVMe storage drives in storage and hyper-converged systems. The BF1500 Controller Card delivers the highest NVMe-oF target performance, reducing TCO and increasing ROI. It can transform existing JBOF systems into NVMe-oF compliant solutions, simply by plugging the card into an existing PCIe slot. -
Page 8: Features And Benefits
PCI Express (PCIe) Uses PCIe Gen 3.0 (8GT/s) and Gen 4.0 (16GT/s) through an x16 edge connector. Gen 1.1 and 2.0 compatible. Up to 100 Gigabit NVIDIA Controllers comply with the following IEEE 802.3 standards: Ethernet – 100GbE / 50GbE / 40GbE / 25GbE / 10GbE / 1GbE –... - Page 9 NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. NVIDIA BF1500 Controller Card effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.otocol.
- Page 10 BlueField BF1500 Controller Card may operate as a co-processor offloading specific storage tasks from the host, isolating part of the storage media from the host, or enabling abstraction of software-defined storage logic using the BlueField Arm cores. On the storage initiator side, BlueField BF1500 card can prove an efficient solution for hyper-converged systems to enable the host CPU to focus on compute while all the storage interface is handled through the Arm cores.
-
Page 11: Operating Systems/Distributions
Operating Systems/Distributions The BF1500 Controller Card is shipped with Linux based Operating System burned on it which includes all needed drivers. For more information, please refer to the BlueField Software User Manual. Connectivity • Interoperable with 1/10/25/40/50/100 Gb/s Ethernet switches •... -
Page 12: Supported Interfaces
Supported Interfaces BF1500 Controller Card Layout and Interface Information The below figures show the component and print sides of BF1500 Controller Card. Each numbered interface that is referenced in the figures is described in the following table with a link to detailed information. ... - Page 13 Item Interface Description "BlueField System-on-Chip (SoC)” Mellanox BlueField SoC 16 cores “Ethernet QSFP28 Interface” Ethernet traffic is transmitted through the BF1500 Controller Card QSFP28 connectors. The QSFP28 connectors allow for the use of modules, optical and passive cable interconnect solutions “PCI Express Interface”...
-
Page 14: Interfaces Detailed Description
Interfaces Detailed Description BlueField System-on-Chip (SoC) BlueField is a Mellanox family of advanced SoC solutions that integrate a coherent mesh of 64-bit Arm v8 A72 cores, a ConnectX network adapter front-end and a PCI Express switch into a single chip. The powerful SoC architecture includes an ARMv8 multicore processor array and enables customers to develop sophisticated applications and highly differentiated feature sets. -
Page 15: Ddr4 Sdram On-Board Memory
DDR4 SDRAM On-Board Memory The BF1500 Controller Card incorporates a single channel DDR4 SDRAM memory with 8 components of 8 bit + ECC (64bit + 8bit ECC), 16GB @2400MT/s. NC-SI Management Interface The BlueField Controller Card enables the connection of a Baseboard Management Controller (BMC) to a set of Network Interface Controller (NICs) for the purpose of enabling out-of-band remote manageability. -
Page 16: Pin Header Interface
3 Pin Header Interface The 3 pin header interface enables access to the Arm console through UART0 through the following pins: • Pin # 1 - UART0 RX • Pin # 2 - GND • Pin # 3 - UART0 TX UART Interface Connectivity The USB to UART cable that supports TTL voltage levels can be used to connect the UART Interface. -
Page 17: Mini Usb Type B Interface
Mini USB Type B Interface The BF1500 Controller Card uses a Mini USB Type B. The USB connector is mounted on the card. The purpose of the USB connector is loading operating system images for the operating systems. In order to use this interface, please use a standard Mini USB Type-B to Mini USB Type A cable. JTAG CoreSight 10 Interface The BF1500 Controller Card uses JTAG CoreSight 10-Pin Header. - Page 18 Note a. 1Hz Blinking amber occurs due to running a beacon command for locating the adapter card.
-
Page 19: Pin Description
Pin Description PCI Express Pins Description The below table lists the PCI Express pins description. For further details, please refer to PCI Express Interface. PCI Express Pin Description Pin # Signal Name Description Pin # Signal Name Description PRSNT1# Mechanical Present JTAG SMCLK Host SMBus... - Page 20 Pin # Signal Name Description Pin # Signal Name Description RSVD REFCLK+ Host Reference Clock REFCLK- Host Reference Clock PETP0 PETN0 PERP0 PERN0 RSVD RSVD PETP1 PETN1 PERP1 PERN1 PETP2 PETN2 PERP2 PERN2 PETP3...
- Page 21 Pin # Signal Name Description Pin # Signal Name Description PETN3 PERP3 PERN3 RSVD RSVD RSVD RSVD PETP4 PETN4 PERP4 PERN4 PETP5 PETN5 PERP5 PERN5 PETP6 PETN6 PERP6...
-
Page 22: Nc-Si Management Interface
Pin # Signal Name Description Pin # Signal Name Description PERN6 PETP7 PETN7 PERP7 PERN07 PRSNT2# Mechanical Present NC-SI Management Interface The below table lists the NC-SI pins description. For further details, please refer to NC-SI Management Interface. NC-SI Management Pins Description Pin # Type Description... - Page 23 Pin # Type Description ARB_OUT Output Network Controller hardware arbitration Power Ground RXD0 Output Receive data Power Ground RXD1 Output Receive data Power Ground CRS_DV Output Carrier Sense/Receive Data Valid Power Ground TXD0 Input Transmit data Power Ground TXD1 Input Transmit data Power Ground...
-
Page 24: Jtag Coresight 10
Pin # Type Description SDA (Not Used) Power Ground SCL (Not Used) Input Power Ground Power Ground Power Ground 3.3V (Not Used) Power 3.3V (Not Used) Power 3.3V (Not Used) Power 3.3V (Not Used) Power JTAG CoreSight 10 The below table lists the JTAG CoreSight 10 pins description. For further details, please refer to JTAG CoreSight 10 Interface. - Page 25 Pin# Signal Name Voltage Domain Description VTREF The Voltage Target Reference pin supplies DSTREAM with the debug rail voltage of the target to match its I/O logic levels. VTREF can be tied HIGH on the target. If VTREF is pulled HIGH by a resistor, its value must be no greater than 100Ω. The Test Mode Select pin sets the state of the Test Access Port (TAP) controller on the target.
-
Page 26: Hardware Installation
Hardware Installation Installation and initialization of the BlueField BF1500 Controller Card require attention to the mechanical attributes, power specification, and precautions for electronic equipment. Safety Warnings Safety warnings are provided here in the English language. For safety warnings in other languages, refer to the... -
Page 27: System Requirements
A system with a PCI Express high power slot that supports 75W is required for installing the card. Airflow Requirements BlueField BF1500 Controller Card is offered with one airflow pattern: from the heatsink to the network ports, as described in the below figure. -
Page 28: Software Requirements
It is prohibited to use port-to-heatsink airflow as it may cause damage to the BlueField BF1500 Controller Card. Please refer to the Specifications section for airflow numbers for each specific card model. Software Requirements • Operating Systems/Distributions section under the Introduction section. -
Page 29: Unpacking The Package
Turn off the power to the system, and disconnect the power cord. Refer to the system documentation for instructions. Before you install the BlueField BF1500 Controller Card, make sure that the system is disconnected from power. (Optional) Check the mounting bracket on the BlueField BF1500 Controller Card. -
Page 30: Installation Instructions
Please note that the following figures are for illustration purposes only. Open the system case. Place the BF1500 Controller Card in an available PCI Express slot. The BlueField BF1500 Controller Card should be installed in a high power PCI Express x8 slot as the card consumes up to 55W. - Page 31 3. Applying even pressure at both corners of the card, insert the BF1500 Controller Card into the PCI Express slot until firmly seated. Do not use excessive force when seating the card, as this may damage the system or the BF1500 Controller Card.
- Page 32 4. When the BF1500 Controller Card is properly seated, the port connectors are aligned with the slot opening, and the BF1500 Controller Card faceplate is visible against the system chassis.
-
Page 33: Cables And Modules
Secure the BF1500 Controller Card with the screw. Close the system case. To uninstall the BF1500 Controller Card card, see Uninstalling the Card. Cables and Modules Cable Installation All cables can be inserted or removed with the unit powered on. To insert a cable, press the connector into the port receptacle until the connector is firmly seated. -
Page 34: Identify The Bf1500 Controller Card In Your System
Determine the correct orientation of the connector to the card before inserting the connector. Do not try and insert the connector upside down. This may damage the BF1500 Controller Card card. Insert the connector into the BF1500 Controller Card card. Be careful to insert the connector straight into the cage. Do not apply any torque, up or down, to the connector cage in the BF1500 Controller Card card. -
Page 35: On Linux
In the Value display box, check the fields VEN and DEV (fields are separated by ‘&’). In the display example above, notice the sub-string “PCI\VEN_15B3&DEV_A2D2/A2D3”: VEN is equal to 0x15B3 - this is the Vendor ID of Mellanox Technologies; and DEV is equal to A2D2 or A2D3- this is a valid Mellanox Technologies PCI Device ID. - Page 36 Verify that the system is powered off and unplugged. Wait 30 seconds. To remove the card, disengage the retention mechanism on the bracket (screws). Holding the BF1500 Controller Card from its center, gently pull the BF1500 Controller Card out of the PCI Express slot.
- Page 37 5. When the port connectors reach the top of the chassis window, gently pull the BF1500 Controller Card in parallel to the motherboard. ...
-
Page 38: Bring-Up And Driver Installation
Bring-Up and Driver Installation It is recommended to upgrade your BlueField product to the latest software and firmware versions in order to enjoy the latest features and bug fixes. Instructions for the BlueField Controller card bring-up and driver installation are located in the BlueField Software documentation at Controller Card Bring- Up and Driver Installation. -
Page 39: Troubleshooting
Troubleshooting General Troubleshooting • Ensure that the BF1500 Controller Card is placed correctly Server unable to find the BF1500 • Make sure the BF1500 Controller Card slot and the BF1500 Controller Card are compatible Controller Card Install the BF1500 Controller Card in a different PCI Express slot •... -
Page 40: Specifications
Specifications MBF1L516B-CSNAT Specifications BlueField SoC BlueField™ G-Series 16 Cores Physical Size: 4.37in. x 6.6 in. (111.15mm x 167.65 mm) Connector: Dual QSFP28 (copper and optical) Protocol Support Ethernet: 100GBASE-CR4, 100GBASE-KR4, 100GBASE-SR4, 50GBASE-R2, 50GBASE-R4, 40GBASE-CR4, 40GBASE-KR4, 40GBASE-SR4, 40GBASE-LR4, 40GBASE-ER4, 40GBASE-R2, 25GBASE-R, 20GBASE-KR2, 10GBASE-LR,10GBASE-ER, 10GBASE-CX4, 10GBASE-KX4, 10GBASE-CR, 10GBASE-KR, SGMII, 1000BASE-CX, 1000BASE-KX, 10GBASE-SR Data Rate: 1/10/25/40/50/100 Gb/s Ethernet PCI Express Gen3.0/4.0: SERDES @ 8.0 / 16.0GT/s, 16 lanes (2.0 and 1.1 compatible) - Page 41 Temperature Operational 0°C to 45°C Non-operational -40°C to 70°C Humidity Environmental 90% relative humidity Airflow Cable Type At 35°C Ambient At 45°C Ambient Passive Cables (Heatsink-to-port) Active 3.5W Cables Safety CB / cTUVus / CE Regulatory CE / FCC / VCCI / ICES / RCM RoHS RoHS compliant a.
-
Page 42: Mechanical Drawing And Dimensions
Mechanical Drawing and Dimensions Mechanical Drawing of the BF1500 Controller Card - Component Side... -
Page 43: Mechanical Drawing Of The Bf1500 Controller Card - Print Side
Mechanical Drawing of the BF1500 Controller Card - Print Side... -
Page 44: Bracket Mechanical Drawing
Bracket Mechanical Drawing... -
Page 45: Finding The Mac On The Adapter Card
Finding the MAC on the Adapter Card Each controller card has a different identifier printed on the label: serial number and the card MAC for the Ethernet protocol The product revisions indicated on the labels in the following figures do not necessarily represent the latest revisions of the cards. -
Page 46: Document Revision History
Document Revision History Date Comments/Changes May. 2023 Updated Specifications - added non-operational storage temperature specifications Mar. 2020 First release... - Page 47 NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.
- Page 48 INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF ANY USE OF THIS DOCUMENT, EVEN IF NVIDIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s aggregate and cumulative liability towards customer for the products described herein shall be limited in...