Nvidia MSTFLINT Manual
  1. Manuals
  2. Brands
  3. Nvidia Manuals
  4. Tools
  5. MSTFLINT
  6. Manual
Nvidia MSTFLINT Manual

Nvidia MSTFLINT Manual

Burning and diagnostics tools
Hide thumbs Also See for MSTFLINT:
Table of Contents

Advertisement

Quick Links

MSTFLINT Package – Firmware
Burning and Diagnostics Tools
User Manual
Rev. 4.14.0-3
Exported on Jul/07/2020 02:21 PM
https://docs.mellanox.com/x/vwHjAQ

Advertisement

Table of Contents
loading

Related Manuals for Nvidia MSTFLINT

Summary of Contents for Nvidia MSTFLINT

  • Page 1 MSTFLINT Package – Firmware Burning and Diagnostics Tools User Manual Rev. 4.14.0-3 Exported on Jul/07/2020 02:21 PM https://docs.mellanox.com/x/vwHjAQ...

  • Page 2: Table Of Contents

    Table of Contents Release Notes ......................7 Package Tools ......................... 7 Flags Dependencies........................ 8 mstflint Supported Operating Systems and Platforms ............8 Supported Flash Types......................11 Supported Mellanox ICs ......................12 Supported Switch Systems Software ................... 12 Changes and New Features....................12 mstflint Bug Fixes in this Version..................
  • Page 3 Burning a Firmware Image................54 Burning the MFA2 Images ..................56 mstflint: Managing an Expansion ROM Image..............56 mstflint: Performing Checksum Calculation on Image/Device ........57 mstflint: Querying the Firmware Image................57 Querying the MFA2 File....................58 mstflint: Setting GUIDs and MACs .................. 58 4th Generation (Group I) Devices................
  • Page 4 Erasing a Sector ......................67 Querying Flash Parameters..................67 mstflint: Comparing the Binary Image................68 mstflint: Firmware Timestamping for Multi-Host Environment........68 Setting a Timestamp on Image................... 68 Querying a Timestamp on Image................68 Resetting a Timestamp on Device ................69 Setting a Timestamp on Device ..................
  • Page 5 Firmware Burning Related Issues ..................96 Secure Firmware Related Issues ..................98 Appendixes ........................100 mstflint - Updating Firmware Using ethtool/devlink and .mfa2 File ........ 100 mstflint User Manual Revision History..............102 Release Notes Document History................105 mstflint Release Notes Change Log History..............105...
  • Page 6 OverviewThe NVIDIA® Mellanox® Firmware Tools (mstflint) package is a set of firmware management and debug tools for Mellanox devices. The document describes mstflint features, tools content and configuration. The documentation here relates to: • Release Notes • User Manual Intended Audience This manual is intended for system administrators responsible for managing and debugging firmware for Mellanox devices.

  • Page 7: Release Notes

    Release Notes These are the release notes for mstflint. mstflint supports Linux operating system. Please see the supported platform table for further details.The tools functionality is identical in all operating systems unless otherwise noted.  Package Tools The following is a list of the available tools in the package, together with a brief description of each tool.

  • Page 8: Flags Dependencies

    Supported Operating Systems and Platforms mstflint is supported on the following platforms: Table Legend:  + (Green) Supported and tested ** (Orange) Supported but not tested in mstflint 4.14.0-3 Supported Operating Systems and Platforms...
  • Page 9 32 Bit x86_64 PPC64LE (QCOM) (Power 9) RHEL6.3 RHEL6.10 RHEL7.2 RHEL7.3 RHEL7.4 RHEL7.5 RHEL7.6 RHEL7.7 RHEL7.8 RHEL8 RHEL8.1 RHEL8.2 (Beta) OEL 6.10 OEL 7.4 OEL 7.7 OEL 8.0 OEL 8.1 Fedora 30 Sles11 SP4...
  • Page 10 32 Bit x86_64 PPC64LE (QCOM) (Power 9) Sles12 SP3 Sles12 SP4 Sles12 SP5 Sles15 Sles15 SP1 Sles15 SP2 (Beta) EulerOS V2.0 SP3 EulerOS V2.0 SP5 Ubuntu 16.04.06 Ubuntu 18.04.02 Ubuntu 18.04.03 Ubuntu 18.04.04 (Beta) Ubuntu 19.04 Ubuntu 19.10 Debian 8.9 Debian 8.11 Debian 9.5 Debian 9.6...

  • Page 11: Supported Flash Types

    (Power 9) Debian 10 VMware ESXi 6.5 Native VMware ESXi 6.7 Native FreeBSD 13.0- CURRENT FreeBSD 12- STABLE FreeBSD 11- STABLE Supported Flash Types mstflint supports the following Flash types. Vendor Flash Family Tested P/N Winbond W25QxxBV W25Q32FVSSIG W25Q32FVSSIGS W25Q32FVSSIGT...

  • Page 12: Supported Mellanox Ics

    Vendor Flash Family Tested P/N Micron N25QOxxx MT25QL128ABA1ESE-0SIT ISSI IS25LPxxx IS25LP128-JBLE SPA# U1323A Cypress S25FL256L S25FL256LDPBHV023 Supported Mellanox ICs The ICs are listed in the following table: IC Group IC Device Group I / 4th Generation • ConnectX®-3 • ConnectX®-3 Pro •...

  • Page 13: Mstflint Bug Fixes In This Version

    Workaround: N/A Keywords: mstflint Discovered in Version: 4.14.0-1 2110890 Description: If there is a json-c library installed on the machine, some of mstflint tools like mstflint and mstfwmanager will not be compiled successfully unless the installed library is removed. Workaround: N/A Keywords: mstlink Discovered in Version: 4.14.0-1...
  • Page 14 Internal Ref. Issue 2084837 Description: Setting the speeds (50GbE and 100GbE) for the new devices (Connect- X 6 and above, Quantum switches and above) requires specifying the number of lanes for the speed: mstlink -d <dev> --speeds [50G_2X | 50G_1X | 100G_2X | 100G_4X] For PRBS mode, to work with PAM4 speeds, use the same speed naming for (50GbE, and 100GbE).
  • Page 15 Keywords: mstconfig Discovered in Version: 4.11.0 1431471 Description: In ConnectX-5 adapter cards, the time-stamp capability using mstflint, is supported only on the device using the "-d" flag, and not on the binary using the "-i" flag. Workaround: Use the “-d” flag to set the time-stamp.

  • Page 16: Introduction

    Introduction The Mellanox Firmware Tools (mstflint) package is a set of firmware management and debug tools for Mellanox devices. mstflint can be used for: • Generating a standard or customized Mellanox firmware image • Querying for firmware information • Burning a firmware image to a single Mellanox device...

  • Page 17: Compilation And Installation

    In-Band device access requires the local IB port to be in the ACTIVE state and connected to an IB fabric. All mstflint tools address the target hardware device using a PCI device. To list all available PCI devices, use the “lspci” command.

  • Page 18: Firmware Generation, Configuration, And Update Tools

    • mstarchive – Binary Files Compression Tool • mstconfig – Changing Device Configuration Tool • mstflint – Firmware Burning Tool • mstfwreset – Loading Firmware on 5th Generation Devices Tool • mstcongestion – Tool for Setting Congestion Mode and Action •...

  • Page 19: Querying The Device

    -D|--image-dir Specified directory instead of default to locate image files. DirectoryName -f|--force Force image update -y|--yes Answer is yes in prompts --no Answer is no in prompts --clear-semaphore Force clear the flash semaphore on the device, No command is allowed when this flag is used. NOTE: May result in system instability or flash corruption if the device or another application is currently using the flash.
  • Page 20 # mstfwmanager --query Examples: Query the device. mstfwmanager -d 09:00.0 --query Querying Mellanox devices firmware ...   Device #1: ---------- Device Type: ConnectX3 Part Number: MCX354A-FCA_A2-A4 Description: ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6 PSID: MT_1020120019 PCI Device Name:...

  • Page 21: Archived Images Content

    # mstfwmanager --query --query-format XML <Devices> <Device pciName="0000:84:00.0" type="ConnectX3Pro" psid="MT_1090111019" partNumber="MCX354A-FCC_Ax"> <Versions> <FW current="2.42.5016" available="N/A"/> <PXE current="3.4.0753" available="N/A"/> </Versions> <GUIDs port1="f452140300347881" port2="f452140300347882"/> <Status>No matching image found</Status> <Description>ConnectX-3 Pro VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE;PCIe3.0 x8 8GT/s;RoHS R6</Description> </Device>...

  • Page 22: Mstarchive Synopsis

    Mfa2 file to parse  The .mfa2 file can be used with ethtool to burn adapter cards firmware. The procedure is described in mstflint - Updating Firmware Using ethtool/devlink and .mfa2 File section. mstconfig – Changing Device Configuration Tool The mstconfig tool allows the user to change some of the device configurations without reburning the firmware.

  • Page 23: Mstconfig Synopsis

    • Connect-IB™: v10.10.6000 or above • Supported devices:  • Adapter cards: ConnectX®-3 /ConnectX®-3 Pro, Connect-IB®, ConnectX®-4/ ConnectX®-4 Lx/ConnectX®-5/ConnectX®-5 Ex/BlueField/ConnectX®-6/ ConnectX®-6 Dx • Switches:  Switch-IB/Switch-IB 2,/Spectrum/Spectrum-2/Quantum • Changing device configurations enabled.  For changes after a successful configuration to take effect, reboot the system. mstconfig Synopsis # mstconfig [Options] <commands>...

  • Page 24: Examples Of Mstconfig Usage

    gen_tlvs_file Generate List of all TLVs. TLVs output file name must be specified. (*) g[en_xml_template] Generate XML template. TLVs input file name and XML output file name must be specified. (*) xml2raw Generate binary configuration dump file from XML file. XML input file name and bin output file name must be specified.

  • Page 25: Setting Device Configuration

     For Array type parameters, the query command will not show a value for it. It will only show you the word "Array" and the range of the array. • For example: HOST_CHAINING_DESCRIPTORS Array[0..7] • To query the fifth element in the array, run: mstconfig -d <device> query HOST_CHAINING_DESCRIPTORS[5] •...

  • Page 26: Using Mstconfig

    # mstconfig -d 41:00.0 reset Reset configuration device 41:00.0? ? (y/n) [n] : y Applying... Done!     -I- Please power-cycle device to load configurations. >mstconfig 41:00.0 query     Device 1: ---------- Device type: ConnectX-3 PCI Device: 41:00.0 Configurations: Next Boot SRIOV_EN True(1)

  • Page 27: Using Mstconfig To Set Sr-Iov Parameters

    # mstconfig 41:00.0 LINK_TYPE_P1=1 LINK_TYPE_P2=1   Device #1: ---------- Device type: ConnectX3Pro PCI device: 41:00.0 Configurations: Next Boot LINK_TYPE_P1 ETH(2) IB(1) LINK_TYPE_P2 ETH(2) IB(1)   Apply Configuration? ? (y/n) [n] : y Applying... Done! -I- Please reboot machine to load configurations.

  • Page 28: Mstconfig Raw Configuration Files

    # mstconfig -d 41:00.0 BOOT_VLAN_P2=3   Device #1: ---------- Device type: ConnectX3Pro PCI device: 41:00.0 Configurations: Next Boot BOOT_VLAN_P2   Apply Configuration? ? (y/n) [n] : y Applying... Done! -I- Please reboot machine to load configurations. mstconfig Raw Configuration Files mstconfig allows applying raw configuration file for a pre-set configuration.

  • Page 29: Mstconfig Commands

    mstconfig Commands mstconfig Backup Command The backup command is used to save the current non-volatile configurations (TLV) in the device into a file in raw TLV syntax so it can be restored anytime using the set_raw command. mstconfig backup command allows backing up the all of the configurations which are related only to the PCI Physical Function associated with the given MST device.

  • Page 30: Mstconfig Xml2Raw Command

    In order to include the nv_kdnet_data configuration in the template, change the 0 to 1, as demonstrated in the following example. Example: nv_kdnet_data   #mstconfig gen_xml_template /tmp/confs.txt /tmp/template.xml Saving output... Done!   #cat /tmp/template.xml <?xml version="1.0" encoding="UTF-8"?> <config xmlns="http://www.mellanox.com/config"> <nv_kdnet_data>   <!-- Legal Values: False/True -->...

  • Page 31: Mstconfig Create_Conf Command

    #mstconfig xml2bin /tmp/template.xml /tmp/confs.bin Saving output... Done! # hexdump -C /tmp/confs.bin 00000000 80 00 00 00 |..| 00000004 mstconfig create_conf Command The create_conf command assists in creating an NV configuration file that can be used for LifeCycle and Secure Firmware Updates purposes. The flow for creating a configuration file is the same as the flow for xml2bin.

  • Page 32: Mstconfig Supported Configurations And Parameters

    mstconfig Supported Configurations and Parameters Feature Parameter Values PCI Settings: Configure SRIOV_EN False: Disable the device PCI settings True: Enable Enables or disables virtualization NUM_OF_VFS=<NUM> 1 to 127 (maximal number of virtual function may be smaller as Sets the number of virtual functions to it depends on the PCI BAR size allocate and available system resources)
  • Page 33 Feature Parameter Values PF_BAR2_ENABLE Values: Unsigned numerical values Enables the physical function bar size. PF_BAR2_SIZE Values: Unsigned numerical values Sets the physical function bar size. DCR_LIFO_SIZE 0 to 16777215 (maximal size may IB Dynamically Connect be smaller depending on the Configure the number of The amount of total DCRs available to firmware)
  • Page 34 Feature Parameter Values 4th Generation Parameters: 0 to 4095 • BOOT_VLAN_P1 • BOOT_VLAN_P2 5th Generation Parameters: • BOOT_VLAN VLAN ID for the network boot LEGACY_BOOT_PROTOCOL 0x0: NONE 0x1: PXE Select boot protocol for legacy BIOS 0x2: ISCSI expansion ROM (FlexBoot) 0x3: BOTH - PXE + iSCSI 0x4: NVME BOOT_RETRY_CNT...
  • Page 35 Feature Parameter Values CLAMP_TGT_RATE_P1 False: Disable CLAMP_TGT_RATE_P2 True: Enable If set, whenever a CNP is processed, the target rate is updated to be the current rate CNP_DSCP_P1 0 to 63 CNP_DSCP_P2 0: Default The DiffServ Code Point of the generated CNP for port1/2 CNP_802P_PRIO_P1 0 to 7 CNP_802P_PRIO_P2...
  • Page 36 Feature Parameter Values RATE_REDUCE_MONITOR_PERIOD_P 0 to 131071 2: Default RATE_REDUCE_MONITOR_PERIOD_P The minimum time between 2 consecutive rate reductions for a single flow. Rate reduction will occur only if a CNP is received during the relevant time interval RPG_AI_RATE_P1 0 to 131071 RPG_AI_RATE_P2 10: Default The rate, in Mbits per second, used to...
  • Page 37 Feature Parameter Values RPG_MAX_RATE_P1, 32 bit integer RPG_MAX_RATE_P2 0: Default (full port speed) The maximum rate, in Mbits per second, at which an RP can transmit. Once this limit is reached, the RP rate limited is released and the flow is not rate limited any more RPG_MIN_DEC_FAC_P1, 1 to 100...
  • Page 38 Feature Parameter Values 4th Generation Parameters: True False FORCE_MODE_P1 FORCE_MODE_P2 4th Generation Parameters: 1: XAUI 2: XFI PHY_TYPE_P1 PHY_TYPE_P2 3: SGMII 4th Generation Parameters: 0: _10G 1: _20G XFI_MODE_P1 XFI_MODE_P2 2: _40G WOL_MAGIC_EN_P1 False: Disable Wake on LAN : Enables/ True: Enable disables Wake on LAN (for ConnectX-3 and ConnectX-3 Pro)
  • Page 39 Feature Parameter Values LLDP_NB_DCBX_P1 False: Disable LLDP Client Setting True: Enable Enable DCBX for the first port (applicable when LLDP_NB_TX_MODE and LLDP_NB_RX_MODE are in ALL mode) LLDP_NB_DCBX_P2 False: Disable True: Enable Enable DCBX for the second port (applicable when LLDP_NB_TX_MODE and LLDP_NB_RX_MODE are in ALL mode) LLDP_NB_RX_MODE_P1...
  • Page 40 Feature Parameter Values LLDP_NB_TX_MODE_P2 • 0x0: OFF – NIC inter- nal LLDP client will not send Select which LLDP TLV will be LLDP frames generated by the NIC • 0x1: MANDATORY – Transmit only manda- tory LLDP BPDU (ChassisID, PortID & TTL) •...
  • Page 41 Feature Parameter Values DCBX_IEEE_P2 False: Disable True: Enable Enable DCBX in IEEE mode. DCBX_WILLING_P1 False: Disable True: Enable Allow the NIC to accept DCBX configuration from the remote peer. DCBX_WILLING_P2 False: Disable True: Enable Allow the NIC to accept DCBX configuration from the remote peer.
  • Page 42 Feature Parameter Values MPFS CONF DUP_MAC_ACTION_P1 / LAST_CFG: The last PF which DUP_MAC_ACTION_P2 added the MAC will receive the traffic Defines the forwarding behavior in MPFS for MACs which are duplicated LOAD_BALANCE: MPFS will load- in more than one PF. balance IPv4 traffic for packets with destination MAC addresses that belong to more than one PF...
  • Page 43 Feature Parameter Values IP_OVER_VXLAN_PORT 0 to 65535 MKEY_BY_NAME 0: False 1: True BOOT_DBG_LOG True ROM DEBUG LEVEL False When this bit is set, FlexBoot will generate debug log at the levels specified by the following parameters. Logs are dumped to the display and to the serial console (if configured).
  • Page 44 Feature Parameter Values BOOT_DBG_LOG_NDRV 0x0: DISABLE 0x1: BASIC Debug level for FlexBoot’s NODNIC 0x2: ADVANCED driver (init, teardown, poll CQ etc) 0x3: ALL BOOT_DBG_LOG_NDRV_CMD 0x0: DISABLE 0x1: BASIC Debug level for FlexBoot’s NODNIC 0x2: ADVANCED Driver interface 0x3: ALL BOOT_DBG_LOG_NDRV_DEV 0x0: DISABLE 0x1: BASIC Debug level for FlexBoot’s NODNIC...
  • Page 45 Feature Parameter Values BOOT_DBG_LOG_TCPIP 0x0: DISABLE 0x1: BASIC Debug level for FlexBoot’s TCP/IP 0x2: ADVANCED stack. 0x3: ALL BOOT_DBG_LOG_TFTP 0x0: DISABLE 0x1: BASIC Debug level for FlexBoot’s TFTP stack. 0x2: ADVANCED 0x3: ALL BOOT_DBG_LOG_UDP 0x0: DISABLE 0x1: BASIC Debug level for FlexBoot’s UDP stack. 0x2: ADVANCED 0x3: ALL BOOT_DBG_LOG_URI...
  • Page 46 Feature Parameter Values UEFI_DEBUG_LOG_BLKIO True False Enable UEFI debug level logs for BlkIo Driver. UEFI_DEBUG_LOG_BM Enable UEFI debug level logs for Boot Manager. UEFI_DEBUG_LOG_CACHE Enable UEFI debug level logs for Memory range cachebility changes. UEFI_DEBUG_LOG_DISPATCH Enable UEFI debug level logs for PEI/ DXE/ SMM Dispatchers.
  • Page 47 Feature Parameter Values ADVANCED_POWER_SETTINGS True POWER CONF False Enable manual override for the default power settings. When Set, additional power settings parameter in NV_POWER_CONF are valid and configurable. When cleared, additional power settings parameter in NV_POWER_CONF are ignored. DISABLE_SLOT_POWER_LIMITER True False When cleared, the device is not allowed to consume more than 25W...
  • Page 48 Feature Parameter Values EXP_ROM_UEFI_x86_ENABLE True False EXP_ROM_UEFI_ARM_ENABLE True False MEMIC_BAR_SIZE Numerical value MEMIC Conf The amount of BAR size assigned for MEMIC. The size in bytes MEMIC_SIZE_LIMIT 0x0: DISABLED 0x1: 256KB The maximum amount of internal 0x2: 512KB device memory that can be consumed 0x3: 1024KB by the MEMIC application.
  • Page 49 Feature Parameter Values ECPF Conf ECPF_ESWITCH_MANAGER Values: EXTERNAL_HOST_PF=0x0,ECPF= ECPF_PAGE_SUPPLIER Values: EXTERNAL_HOST_PF=0x0,ECPF= FLEX PARSER Conf FLEX_PARSER_PROFILE_ENABLE Values: Numerical values FLEX_IPV4_OVER_VXLAN_PORT Values: Numerical values Safe Mode Conf SAFE_MODE_THRESHOLD Values: Numerical values SAFE_MODE_ENABLE 0: False 1: True NVME (Alpha level) NVME_EMULATION_ENABLE False: Disable True: Enable NVME_EMULATION_NUM_PF Values: Unsigned numerical...

  • Page 50: Mstflint - Firmware Burning Tool

    Note that the textual values are case insensitive (either “True” or “true” are accepted). mstflint – Firmware Burning Tool The mstflint (Flash interface) utility performs the following functions: • Burns a binary firmware image to the Flash device attached to an adapter or a switch device.
  • Page 51 -banks <banks> Set the number of attached flash devices (banks)  -blank_guids Burn the image with blank GUIDs and MACs (where applicable). These values can be set later using the "sg" command (see details below). Commands affected: burn -clear_semaphore Force clear the flash semaphore on the device. No command is allowed when this flag is used.
  • Page 52 NOTE: This flag is intended for advanced users only. Running in this mode may cause the firmware to hang. Run a quick query. When specified, mstflint will not perform full image integrity checks  during the query operation. This may shorten execution time when running over slow ...

  • Page 53: Command Parameters

    -use_image_guids Burn (guids/uids/macs) as appears in the given image. Commands affected: burn -use_image_ps Burn vsd as appears in the given image - do not keep existing VSD on flash. Commands affected: burn -use_image_rom Do not save the ROM which exists in the device. Commands affected: burn Version info.

  • Page 54: Mstflint: Burning A Firmware Image

    Burning a Firmware Image The mstflint utility enables you to burn the Flash from a binary image.To burn the entire Flash from a raw binary image, use the following command line: # mstflint -d <device> -i <fw-file> [-guid <GUID> | -guids <4...
  • Page 55 To burn a firmware image: Update the firmware on the device, keeping the current GUIDs and VSD. (Note: This is the common way to use the tool. # mstflint -d 41:00.0 -i fw-4099-2_42_5000-MCX354A-FCB_A2.bin burn Update the firmware on the device, specifying the GUIDs to burn.

  • Page 56: Burning The Mfa2 Images

    The device flash MUST have all relevant device information (signatures, PSID, VPD, DEV_INFO, MFG_INFO, etc.) valid since MFA2 format does not have that information and without the burn process will fail. #mstflint -d <device> -i <mfa2 file> --psid <PSID string> (optionally) -- latest_fw (optionally) –silent (optionally) b (or burn) •...

  • Page 57: Mstflint: Performing Checksum Calculation On Image/Device

    For example: the sections that are changed when performing a firmware upgrade. To perform a checksum on the flash, run the following command line:# mstflint -d <mst device> checksumTo perform a checksum on a firmware image, run the following command line: # mstflint -i <image file>...

  • Page 58: Querying The Mfa2 File

    • Default Update Method" field in Query Full output:{This field reflect the method which mstflint will use in order to update the device. The user can enforce a different method using the –no_fw_ctrl or the –ocr flags.The default methods are: •...
  • Page 59 VSD: PSID: MT_1090120019 # mstflint -d 41:00.0 -guid 0x452140300abadaba -mac 0x300abadaba sg -W- GUIDs are already set, re-burining image with the new GUIDs ... You are about to change the Guids/Macs/Uids on the device: New Values Current Values...

  • Page 60: 5Th Generation (Group Ii) Devices

    0002c903002ef508 Base MAC1: 0002c92ef500 Base MAC2: 0002c92ef508 Image VSD: Device VSD: PSID: MT_1240110019 # mstflint -d 41:00.0 -uid 0000000000000001 -ocr sg guids_num=2 step_size=1 -W- Firmware flash cache access is enabled. Running in this mode may cause the firmware to hang.
  • Page 61 MT_1240110019 Change GUIDs and MACs for the ConnectX®-4 device:  # mstflint -d 41:00.0 -guid e41d2d0300570fc0 -mac 0000e41d2d570fc0 -ocr sg -W- Firmware flash cache access is enabled. Running in this mode may cause the firmware to hang. Updating GUID section - OK...

  • Page 62: Preparing A Binary Firmware Image For Pre-Assembly Burning

    In this method, the image is generated with blank GUIDs and CRCs. The GUIDs are set after the device is assembled using the mstflint "sg" command. To set GUIDs take less than 1 second when running on an image with blank GUIDs (through a PCI device).

  • Page 63: Mstflint: Verifying The Firmware Image

    -i ./fw-ConnectX3-rel.bin -guid 0x0002c9030abcdef0 -mac 0x0002c9bcdef1 -striped_image sg (Optional) After assembly, query the image on flash to verify there are no GUIDs on the device. sg# mstflint -i ./fw-ConnectX3-rel.bin -striped_image q Image type: FW Version: 2.31.5050 FW Release Date: 4.5.2014 Device ID:...

  • Page 64: Comparing The Binary Image

    MFA2 archive does not have the dTOC information by its definition, the binary comparison will ignore the device specific sections on the device. #mstflint -d <device> -i <fw image> --silent (optional) bc (or binary_compare) Verifying MFA2 Archive Binary verifying of MFA2 archive enables the user to verify that a given MFA2 archive contains the image that matches the given device.

  • Page 65: 4Th Generation Devices

    Access the HW while HW access is disabled: # mstflint -d 41:00.00 E- Cannot open 41:00.0: HW access is disabled on the device. E- Run "mstflint -d 41:00.0 hw_access enable" in order to enable HW access. Enable HW access: # mstflint -d 41:00.0...

  • Page 66: Mstflint: Flash Operations

    # mstflint -d 41:00.0 0x20 Writing a dword to Flash To write one dword to Flash memory, use the following command line: # mstflint -d <device> ww addr data where: device The device the dword is written to. addr The address of the word to write.

  • Page 67: Writing A Dword To Flash Without Sector Erase

    0x10008 0x5a445a44 Writing a dword to Flash Without Sector Erase To write one dword to Flash memory without sector erase , use the following command line: # mstflint -d <device> wwne addr data where: device The device the dword is written to..

  • Page 68: Mstflint: Comparing The Binary Image

    Setting a Timestamp on Image In order to set a timestamp on an image, run: # mstflint -i ./fw-4115.bin timestamp set [UTC time]  The user can either specify a combined date and time timestamp in UTC which conforms to ISO 8601, or let the tool use the machine’s time for the timestamp.

  • Page 69: Resetting A Timestamp On Device

    In case it is not possible to modify the firmware image, it is possible to set the timestamp directly on the device by specifying the timestamp and firmware version tied to it. # mstflint -d /41:00.0/mt4115_pciconf0 timestamp set <UTC time> <Firmware version>...

  • Page 70: Mstflint Limitations

    Lack of timestamp in both image and device will cause no checks to be performed. mstflint Limitations • When running mstflint via an MTUSB-1 device, a burn/query command may take up to 45 minutes to complete. • To accelerate the burn process add the flag -no_flash_verify to the command line which skips the flash verification step.

  • Page 71: Disabling/Enabling Access To The Hardware

    Make sure you have INI and mlx files suitable for the device. Both files are available for download at: http://www.mellanox.com/page/custom_firmware_table Add cr_protection_en=true under [HCA] section in the INI file. Burn the image on the device using mstflint: # mstflint -d 41:00.0 -i fw-4099.secure.bin b For changes to take effect, reboot is required.

  • Page 72: Key Loss Recovery

    Re-burn firmware Remove the flash-not-present jumper. Reboot the machine Re-set the hardware access key mstflint: Secure Firmware Update  Secure Firmware Update is supported only on ConnectX-4 onwards adapter cards and as of mstflint v4.10.0-3. . A “Secure firmware update” is the ability of a device to verify digital signatures of new firmware binaries, in order to assure that only officially approved versions can be installed from the host, the...
  • Page 73 Tool Flow Secure FW With CS Token Blocked Command Set GUIDs Working with controlled fw Working with controlled fw update update Verify Working partially (BOOT Working partially (BOOT image) image) Set DV INFO: SET Not supported in Secure Not supported in Secure MFBA MFG, SET VSD, VPD ROM OPS: BROM,...

  • Page 74: Setting A "Public Keys" Section In A Binary Image File

    # mstflint -i /tmp/image.bin set_forbidden_versions forbidden_versions.bin Secure Firmware Implications on Burning Tools When Secure Firmware is enabled, the mstflint output slightly changes due to the differences in the underlying NIC accessing methods. Some functionalities may be restricted according to the device security level.

  • Page 75: Re-Signing A Binary Image File

    Files.If the security type permits legacy flash access commands, the --no_fw_ctrl flag can be used to command the mstflint to work in the non firmware controlled mode. This means that all the non-secure functionality will be supported using this flag, and the burn flow will work without requiring a signed image.Example:...
  • Page 76 To generate 4096 bits public keys, please follow the same steps as above, but use "file_public_key_4096" instead of "file_public_key".  For further information, see mstconfig xml2bin Command. Set the key's binary file in the firmware image using the mstflint set_public_keys command.  For further information, see Setting a “Public Keys” Section in a Binary Image File.

  • Page 77: Mstfwreset - Loading Firmware On 5Th Generation Devices Tool

    To sign with a 2048 bits private key only, make sure that the firmware image does not contain a 4096 bits key signature. Run the mstflint set_public_keys command with a 4096 bits keys section filled with zeros and then sign with a 2048 bits private key.

  • Page 78: Reset Levels And Types

    -m|--mst_flags Provide mst flags to be used when invoking mst restart step. For example: -- MST_FLAGS mst_flags="--with_fpga" -y|--yes  Answer “yes” on prompt --skip_driver| -s Skip driver start/stop stage (driver must be stopped manually) -v|--version Print tool version -h|--help  Show help message and exit --skip_fsm_sync ...

  • Page 79: Mstfwreset For Smartnics

    $ lspci -d 15b3: 08:00.0 Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5] 08:00.1 Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5] 0e:00.0 Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5] 0e:00.1 Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5] * All PCI devices above are related to the same NIC  ...

  • Page 80: Mstfwreset Limitations

     • When running the reset command without specifying a reset level the minimal reset level will be performed. • When running the reset command without specifying a reset type the default reset type would be 0 (Full chip reset). To reset a device with a specific reset level to load new firmware, use the following command line: # mstfwreset -d 41:00.0...

  • Page 81: Tool Requirements

    Tool Requirements • Firmware version ConnectX-4 Lx: 14.23.1020 or later  mstcongestion Synopsis # mstcongestion [option] [-d|--device <PCI DEVICE>] [--mode <MODE>] [--action <ACTION>] [-q|-- query] [-h|--help] [-v|--version] where: -d|--device <PCI DEVICE> Mellanox PCI device address --mode <MODE>   Set Mode, options are: [aggressive | dynamic] --action <ACTION>  ...
  • Page 82 --disable_rshim  When TRUE, the host does not have an RSHIM function to access the embedded CPU registers mstconfig_name: HOST_DISABLE_RSHIM --disable_tracer When TRUE, the host will not be allowed to own the Tracer mstconfig_name: HOST_DISABLE_TRACER_OWNER --disable_counter_rd When TRUE, the host will not be allowed to read Physical port counters ...

  • Page 83: Debug Utilities

    Debug Utilities This section contains: • mstfwtrace Utility • mstregdump Utility • mstreg Utility • mstlink Utility • mstresourcedump Utility mstfwtrace Utility The mstfwtrace utility extracts and prints trace messages generated by the firmware running on 5th generation (Group II) devices iRISCs.  These trace messages inform developers of software drivers about internal status, events, critical errors, etc.

  • Page 84: Mstregdump Utility

    • Connect-IB®, ConnectX®-4, ConnectX®-4 Lx, Switch-IB, Switch-IB 2, Spectrum, Quantum: iRISC names: [all] • ConnectX®-5, ConnectX®-5 Ex, BlueField, ConnectX®-6, Spectrum 2 iRISC names: [all] • Trace classes: DEBUG_INIT, INIT, ICM, ICM_FREE_LIST, HOST_MNG, CMD_IF, PHY_IB, PHY_RX_ADAP, PHY_EYE_OPN, PHY_COMMON, PHY_MANAGER, PWR, FLR, ICM_ACCESS, MAD, RXT_CHECKS, I2C, TRANSPORT, FW_LL, RX_ERRORS, CMD_DRIVER, PROFILING, MANAGEMENT, FLASH, STEERING, IFARM, ICMD, PCI, DC_CLEANUP, PHY_ETH, VIRT Example:...

  • Page 85: Mstreg Usage

    Registers can be get/set in unknown (RAW) mode by providing register ID and length.  Unknown (RAW) mode is risky as no checks are performed, please consult with Mellanox support before using it. mstreg Usage mst driver must be started prior to running mstreg tool. Some access registers depend on setup configuration such as link up/down.
  • Page 86 mstreg -d 41:00.0 --show_regs Available Access Registers ================================================================================ ======================== CWTP CWTPM MCIA MLCR MPCNT MPEIN NCFG PAOS PDDR PMDR PMLP PPAOS PPCNT PPLM PPLR PPRT PPTT PTAS PTYS ROCE_ACCL SBCM SBDCR SBPM SBPR SBSR SLRG SLRP SLTP Query a single access register (PAOS): mstreg -d 41:00.0 --show_reg PAOS...

  • Page 87: Mstlink Utility

    mstreg -d 41:00.0 --reg_name PAOS --indexes "local_port=0x1,swid=0x0" --yes -- "e=0x1" You are about to send access register: PAOS with the following data: Field Name | Data ============================= oper_status 0x00000001 admin_status 0x00000001 local_port 0x00000001 swid 0x00000000 0x00000001 0x00000000 0x00000000 =============================   Do you want to continue ? (y/n) [n] : y...

  • Page 88: Mstlink Usage

    mstlink Usage To run mstlink: mstlink [OPTIONS] where: Options: -h |--help Display help message. -v |--version   Display version info. -d |--device <device> Perform operation for a specified mst device -p |--port <port_number>  Port Number --port_type <port_type> Port Type [NETWORK(Default)/PCIE/OOB] --depth <depth> Depth level of the DUT of some hierarchy (valid for PCIe port type only) --pcie_index ...
  • Page 89 --serdes_tx <params> Configure Transmitter Parameters [polarity,ob_tap0,...] --serdes_tx_lane <transmitter_lane> Transmitter Lane to Set (Optional - Default All Lanes) --database   Save Transmitter Configuration for Current Speed Permanently (Optional) --test_mode <prbs_mode> Physical Test Mode Configuration [EN(enable)/ DS(disable)/TU(perform tuning)] --rx_prbs <rx_prbs_mode> RX PRBS Mode [PRBS31(Default)/PRBS7/...] (Optional - Default PRBS31) --tx_prbs <tx_prbs_mode>...

  • Page 90: Tool Usage With Nic Vs. Switch (-P Flag)

    Configure FEC: mstlink -d <device> -p <port_number> --fec RS Configure Port for Physical Test Mode: mstlink -d <device> -p <port_number> --test_mode EN (--rx_prbs PRBS31 --rx_rate 25G --tx_prbs PRBS7 --tx_rate 10G) Perform PRBS Tuning: mstlink -d <device> -p <port_number> --test_mode TU Configure Transmitter Parameters (on lane, to database): mstlink -d <device>...

  • Page 91: Mstresourcedump Utility

     • Any mstlink command for switch should include the "-p" flag to address the specific port in the switch. • When working with the NIC, if an MTUSB is used for communication with the Mellanox NIC, to address port 2, use mstlink -d /dev/mst/mt4115_pciconf0 -- gvmi_address <0xAddress>.

  • Page 92: Mstresourcedump Dump Usage

    An example of how to run the query command:  # mstresourcedump query --device 04:00.0 __________________________________________________________________   Segment Type - 0x1300 (FULL_EQC) Dump Params Applicability Special Values -------------------------------- -------------- -------------- index1 (EQN) Mandatory num-of-obj1 index2 (N/A) num-of-obj2 __________________________________________________________________ __________________________________________________________________   Segment Type - 0x1000 (FULL_QPC) Dump Params...

  • Page 93: Examples Of How To

    -d, --device The device name --segment The segment to dump Examples of how to: • run the dump command:  # mstresourcedump dump --device 04:00.0 --segment 0x1200 --index1 0x404 depth Found segments: ------------------------------------------- Segment Type: 0xfffe Segment Size: Bytes Segment Data: 0x0004FFFE 0x00000000 0x00000000 0x101A0111 ------------------------------------------- Segment Type:...

  • Page 94: Troubleshooting

    PCI device Tools PCI semaphore might be Run the following command: with the following errors: locked due to unexpected # mcra -c <mst_pci_device>...

  • Page 95: Installation Related Issues

    Issue Cause Solution When Querying for current The current firmware on Update to the latest firmware configuration on ConnectX-3/ the device does not ConnectX-3Pro, some of the support showing the parameters are shown as “N/ device's default A” configuration After resetting configuration Firmware loads the default Reboot the server using the tool on 5th...

  • Page 96: Firmware Burning Related Issues

    Connect-IB driver: /etc/init.d/openibd trying to query/burn a Connect- adapter. stop. IB device: Add “-ocr” option to the 'mstflint' command. -E- Cannot open Device: 41:00.0. B14 Operation not For example: permitted mstflint -d 41:00.0 -ocr q MFE_CMDIF_GO_BIT_BUSY...
  • Page 97 -d <device> -E- Burning FS3 image failed: stamped with an older ts reset mstflint -i <image> ts Stamped FW version timestamp reset missmatch: 12.16.0212 differs from 12.16.0230...

  • Page 98: Secure Firmware Related Issues

    Issue Cause Solution Burning the image on The image was changed without Run 'mstflint -i <img.bin> sign', Controlled FW (default update calculating the new digest on it and retry. method: fw_ctrl in 'mstflint -d with 'mstflint -i <img.bin> sign'. <device> query full' output),...
  • Page 99 • The FW image is not secured • The key is not applicable Secure Firmware fails when using mstflint mstflint brom and drom brom and drom commands. commands are not supported. When the CR space is in read only mode,...

  • Page 100: Appendixes

    Appendixes • mstflint - Updating Firmware Using ethtool/devlink and .mfa2 File mstflint - Updating Firmware Using ethtool/ devlink and .mfa2 File In order to flash the firmware on the device using ethtool, you need to prepare a .mfa2 firmware file using the mstarchive tool - see mstarchive –...
  • Page 101 # lspci -n | grep 15b3   04:00.0 0207: 15b3:1017 04:00.1 0207: 15b3:1017     # mstflint -d 04:00.0 Image type: FW Version: 16.26.0292 FW Version(Running): 16.25.1020 FW Release Date: 9.9.2019 Product Version: 16.26.0292 Rom Info: type=UEFI version=14.19.13 cpu=AMD64 type=PXE version=3.5.802...

  • Page 102: Mstflint User Manual Revision History

    User Manual Revision History Revision Date Description 3.8 (SW v4.14.0-1) July 06, 2020 Added Querying the MFA2 File. 3.7 (SW v4.13.3) December 12, • Added a new registry key to mstreg Utility 2019 • Added the following section: •...
  • Page 103 Revision Date Description Dec. 2, 2018 • Added the following sections: • mstarchive - Binary Files Compression Tool • Using mstconfig to Split a Port in a Remotely Managed Switch • mstcongestion - Tool for Setting Congestion Mode and Action •...
  • Page 104 Revision Date Description Mar. 1, 2018 • Added the following sections: • For 5th Generation Devices • Updated the following sections: • Querying the Device Configuration • mstconfig create_conf Command • Signing Binary Image Files • mget_temp Usage • mstlink Usage •...

  • Page 105: Release Notes Document History

    Release Notes Document History mstflint Release Notes Change Log History Component / Description Operating Tool System Rev. 4.14.0-1 mstresourcedu Added support for "--virtual-hca-id" command. Now the tool can Linux provide info on the virtual HCA (host channel adapter, NIC) ID.
  • Page 106 Burning MFA2 Enables the user to extract (i.e. unzip) 4MB images from MFA2 Images archive, that matches the device type and device PSIB. If there are more than one matching image, the user may use – flag and burn the latest firmware, or choose the required latest image from the user menu.
  • Page 107 Exposes supported access registers, and allows users to obtain information regarding the registers fields and attributes, and to set and get data with specific register. mstflint Added image-reactivation feature which re-assigns the image signature to the previous image (in case the new image is faulty) enabling "fwreset"...
  • Page 108 General Removed the COMFIG COMPACT definition.  Linux General Added support for libibmad 12. Linux mstconfig Renamed the BOOT_RETRY_CNT1 parameter to BOOT_RETRY_CNT. Bug Fixes mstflint Bug Fixes History Rev. 4.11.0-5 mstconfig Added the below new params: • pci_atomics_disabled_ext_atomics_enabled • pci_atomics_enabled_ext_atomics_enabled •...
  • Page 109 Aligned all python scripts to unix encoding. mstprivhost Added mstprivhost tool to set host privilege configurations. mstflint Added missing Rom info attribute for legacy FW qq query. Renamed "--next_boot_fw_ver" to "--flashed_version" flag. This flag queries the flashed version. Note that the previous flag is supported to keep compatibility.
  • Page 110 .deb Package As of mstflint v4.12.0, the name of *.deb files will be changed from “mstflint-<version> . amd64.deb” to “mstflint_<version>_amd64.deb” Name e.g., from mstflint-4.10.0-104.amd64.deb to mstflint_4.10.0-104_amd64.deb Supported Added support for Mellanox Quantum switch systems and ConnectX-6 Devices Ready adapter cards.
  • Page 111 mstconfig Added the following new parameters: • FLEX_PARSER_PROFILE_ENABLE • ECPF_ESWITCH_MANAGER • ECPF_PAGE_SUPPLIER • SAFE_MODE_ENABLE • SAFE_MODE_THERSHOLD • BOOT_UNDI_NETWORK_WAIT Bug Fixes MFT Bug Fixes History Linux Rev. 4.9.1 mstfwreset Added support for mstfwreset in Power9 platforms. Linux Rev. 4.9.0 mstfwreset Added support for a hot swap (or hot plug) of the PCIe slot. Linux Secure Added support for Secure Firmware Update to ConnectX-4 adapter...
  • Page 112 Linux Added a flag to enforce working in a non-secure mode, if available (according to security type). Added expansion ROM CPU architecture to the mstflint query when the expansion ROM is available. mstlink Added a new tool that displays and configures port related data at the physical layer.
  • Page 113 General Added support for Innova IPsec 4 Lx EN /Innova Flex 4 Lx EN Linux mstflint package size has been reduced in Linux by separating the Linux / architecture specific RPMs, and in ESXi, by moving relevant tools to ESXi the OEM package.
  • Page 114 Added support for PPTT, PPRT and PPAOS access registers in switches. mstflint Added support for viewing and changing OEMs’ device flash parameters using an IB device when using mstflint. Rev. 4.4.0 mstfwreset Added support for mstfwreset in PowerPC Linux...
  • Page 115 Build Created one mstflint package for all 64 bits FreeBSD OSs FreeBSD mstfwmanager Removed support for mstfwmanager_pci tool (it is deprecated), Linux _pci since all the Linux tools can work without a kernel now. When required, use mstfwmanager instead. mcra Added support for clearing VSEC PCI semaphore by the mcra tool.
  • Page 116 Added the option to display the configuration’s default values. mstflint Added support to calculate checksum on selected sections in the firmware image. Added the option to attach a timestamp to the firmware image. Burning Tools Improved firmware burn performance in livefish mode on 5th generation devices.
  • Page 117 cpld_update Added the cpld_update tool to the OEM package Linux mstfwreset Added support for resetting the firmware Windows/ FreeBSD fwtrace Added support in FreeBSD FreeBSD Burning Tools This version supports new ConnectX-4/Connect-IB firmware version format (MM.mm.ssss). It also enables upgrade of older firmware version format: MM.mmmm.ssss Rev.
  • Page 118 Added mstconfig tool for changing non volotaile configuration on Windows device Burning Tools Added support for micron flash in mstflint and updated production Windows burn flow on Connect-IB Rev. 3.6.0 mstconfig Added mstconfig tool for changing non volatile configuration on...
  • Page 119 Redesigned the mstflint and wqdump utility to make their look and Linux/ wqdump feel more user friendly Windows mstflint Added support for brom in Connect-IB® Linux/ Windows mstmdio Added support for the mstmdio utility Linux mstfwmanager Added support for Connect-IB...
  • Page 120 It is no longer required to run mst start/stop when using Windows Winmstflint tools. The service is automatically loaded/unloaded when an mstflint tool is running. The mst service installation was removed from the setup Added support for SwitchX® silicon devices...
  • Page 121 Windows • The ROM section in the image now contains multiple boot Windows images. Therefore mstflint was modified to display information for all of the images in the ROM section. • Added support to display/burn UEFI ROM/ Added an option to set the VSD and GUIDs in a binary image file.
  • Page 122 The ROM section in the image now contains multiple boot images. Linux Therefore the mstflint was modified to display information for all of the images in the ROM section Added support to display/burn UEFI ROM...
  • Page 123 Added the options: --without-image-generation, --disable-dc, and Linux installation --without-kernel which allow for a partial installation in order to change avoid problems with SW dependencies Now allows a non-root user to prepare mstflint RPMs Linux Added Mellanox ConnectX®-2 and BridgeX® support Linux/ Windows...

  • Page 124: Mstflint Bug Fixes History

    Keywords: mstlink Discovered in Version: 4.13.0 Fixed in Release: 4.14.0-1 1797470 Description: Fixed an issue that prevented the mstflint tools from recognizing the second port on dual port devices. Keywords: mstflint, dual port devices Discovered in Version: 4.12.0 Fixed in Release: 4.13.0...
  • Page 125 Discovered in Version: 4.11.0 Fixed in Release: 4.13.0 1655224 Description: Decreased mstflint query timeout from 80 seconds to 8 seconds. In case the tool does not get a response from the device after 8 seconds, the following error message is displayed: "Cannot open Device: /dev/mst/mt4117_pciconf0.
  • Page 126 Internal Issue Ref. No. Keywords: mstfwreset Discovered in Version: 4.10.0 Fixed in Release: 4.11.0 1213983 Description: Connect-IB function per port (FPP_EB) is not exposed at mstconfig. Keywords: mstfwreset, Connect-IB Discovered in Version: 4.7.0 Fixed in Release: 4.9.0 1064918/ Description: mstfwreset does not load the firmware properly on a Socket-Direct card.
  • Page 127 Internal Issue Ref. No. 795226/ Description: Occasionally, mst tools (driver mode) do not function after running mstfwreset in PowerPC machines. 795657/ 862607 Keywords: mstfwreset Discovered in Release: 4.4.0 Fixed in Release: 4.6.0 795756/ Description: mstfwreset disables and enables all Mellanox devices’ Network Interfaces when resetting the firmware on a device that at least one of its network 795916 interfaces is up.
  • Page 128 NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.
  • Page 129 Terms of Sale for the product. Trademarks NVIDIA, the NVIDIA logo, and Mellanox are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the respective companies with which they are associated.

This manual is also suitable for:

Mellanox mstflint

Table of Contents