Nvidia DGX-2 SYSTEM User Manual page 42

Device: 0000:e6:00.0
LINK_TYPE_P1
Device #8:
Device type: ConnectX4
Device: 0000:58:00.0
LINK_TYPE_P1
Device #9:
Device type: ConnectX5
Device: 0000:86:00.0
LINK_TYPE_P1
LINK_TYPE_P2
Switching the Port from Ethernet to
5.8.4
InfiniBand
Make sure that you have started the Mellanox Software Tools (MST) as explain in the
section Starting the Mellanox Software
change.
Change the configuration for the network cluster ports to InfiniBand by setting
1.
LINK_TYPE_P1=1 for each port.
The following example configures all 8 network cluster ports.
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf0 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf1 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf2 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf3 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf4 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf5 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf6 set LINK_TYPE_P1=1
~$ sudo mlxconfig -y -d /dev/mst/mt4119_pciconf7 set LINK_TYPE_P1=1
Verify the configuration changes have been applied.
2.
$ sudo mlxconfig query | egrep -e Device\|LINK_TYPE
Device #1:
Device type: ConnectX4
Device: 0000:bd:00.0
LINK_TYPE_P1
Device #2:
Device type: ConnectX4
Device: 0000:b8:00.0
LINK_TYPE_P1
Device #3:
Device type: ConnectX4
Device: 0000:3a:00.0
LINK_TYPE_P1
Device #4:
Device type: ConnectX4
Device: 0000:e1:00.0
LINK_TYPE_P1
DGX-2 System User Guide
ETH (1)
ETH (1)
ETH(2)
ETH(2)
Tools, and have identified the correct ports to
IB(1)
IB(1)
IB(1)
IB(1)
Network Configuration
42