Host As Root Port On X4 Pcie Lane Peripherals - Nvidia BlueField-3 User Manual

Networking platform

Hide thumbs Also See for BlueField-3:

Table of Contents

5.12.1 Host as Root Port on x4 PCIe Lane Peripherals

In this scenario, the x16 PCIe Goldfingers of the BlueField-3 DPU serve as endpoints to the host

server (default), while the additional x16 PCIe lanes are accessible via the Cabline CA-II Plus

connector, bifurcated into four PCIe links, where each link comprises x4 PCIe lanes.

In this configuration the host server assumes the role of the Root Port for downstream devices

connected to the Cabline CA-II Plus connector. These downstream devices are exposed to the host

server on its PCIe via the internal BlueField-3 DPU PCIe switch.

As seen in the below visual representation of this configuration, the host functions as the Root Port,

branching into four PCIe links on the Cabline CA-II Plus connector, with each link featuring a

bifurcation of x4 PCIe lanes.

Important Notes:

•

mlxconfig can be configured either through the host in NIC Mode and DPU Mode, or

directly from the DPU's Arm running OS.

•

This configuration is persistent even following resets and NIC firmware updates.

The required set of configurations to implement this bifurcation is outlined below.

mlxconfig -d <device> s

PCI_SWITCH0_UPSTRAEM_PORT_BUS=0

PCI_SWITCH0_UPSTRAEM_PORT_PEX=0

PCI_BUS00_HIERARCHY_TYPE=0

PCI_BUS00_WIDTH=5

PCI_BUS00_SPEED=4

PCI_BUS10_HIERARCHY_TYPE=1

PCI_BUS10_WIDTH=3

PCI_BUS10_SPEED=4

PCI_BUS12_HIERARCHY_TYPE=1

PCI_BUS12_WIDTH=3

Table of Contents