Nvidia DGX-2 System Service Manual page 51

e6:00.0 Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5]
The dual port cards are identified by bus ID 86. Look for all other cards. If eight cards
(those other than bus ID 86) are not reported, then the card was not installed properly and
should be reseated.
If a card other than the officially supported Mellanox family of adapters appears, contact
NVIDIA Enterprise Support.
2. Verify the firmware version.
cat /sys/class/infiniband/mlx5*/fw_ver
$
Example output:
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
The latest InfiniBand firmware version supported for each DGX OS Server release is as
follows:
‣
Release 4.x: Firmware version 12.23.1020
3. If you need to update the firmware, follow these steps:
a). Initiate the firmware update.
sudo /opt/mellanox/mlnx-fw-updater/mlnx_fw_updater.pl
$
The script will check the firmware version of each card and update where needed.
If the firmware is updated for any card, you will need to reboot the system for the
changes to take effect.
b). Reboot the system if instructed.
c). After rebooting the system, verify that all the Mellanox InfiniBand cards are using the
current firmware.
cat /sys/class/infiniband/mlx5*/fw_ver
$
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
12.23.1020
4. Verify the physical port state for the InfiniBand cards.
ibstat
$
In the output text, verify that the Physical State for each card with a cable connection is
LinkUp and that the port for the card is configured with a GUID. The following example
output shows one card in a non-connected state, and the remaining cards in a connected
state. Relevant text is highlighted in bold.
CA 'mlx5_0'
CA type: MT4119
Number of ports: 1
Firmware version: 12.23.1020
Hardware version: 0
Node GUID: 0x248a0703000de288
System image GUID: 0x248a0703000de288
DGX-2 System
ConnectX-5 Card Replacement
DU-09224-001 _v09 | 45