Starting The Mellanox Software Tools And Determining The Current Port Configuration; Switching The Port Configuration - Nvidia DGX A100 User Manual

InfiniBand, then the external switch should be an InfiniBand switch with the corresponding
InfiniBand cables.
Likewise, if the port configuration is set to Ethernet, then the switch should also be Ethernet.
The DGX A100 is also equipped with one (and optionally two) dual-port connections typically
used for network storage and configured by default for Ethernet. These can also be configured
for InfiniBand.
Note: On the dual-port cards, if one of the ports is configured for Ethernet and the other port is
configured for InfiniBand, the following limitations apply.
•
FDR is not supported on the InfiniBand port (port 1 or 2).
•
If port 1 is InfiniBand, then port 2 (Ethernet) does not support 40 GbE/10GbE.
•
If port 1 is Ethernet, then port 2 (InfiniBand) does not support EDR.
7.7.1 Starting the Mellanox Software Tools and
Determining the Current Port Configuration
Make sure that the Mellanox Software Tools services are started.
$ sudo mst start
To determine the current port configuration, enter the following:
$ sudo mlxconfig -e query | egrep -e Device\|LINK_TYPE
The following example shows the output for one of the port devices, showing the device path
and the default, current, and next boot configuration.
Device #2:
Device type: ConnectX6
Device: /dev/mst/mt4123_pciconf8
Configurations:
* LINK_TYPE_P1 IB(1)
IB(1) indicates the port is configured for InfiniBand.

ETH(2) indicates the port is configured for Ethernet.

Determine the Device path bus numbers for the slot number of the port you want to configure.
See the diagram and table in "Configuring Network Proxies" for the mapping.
7.7.2

Switching the Port Configuration

Make sure that you have started the Mellanox Software Tools (MST) services as described in
"Starting the Mellanox Software Tools and Determining the Current Port Configuration" and
have identified the correct ports to change.
DGX A100 System
Default
IB(1)
Current Next Boot
IB(1)
Network Configuration
DU-09821-001_v06 | 47