Summary of Contents for Mellanox Technologies Mellanox SX1018
Page 1
Mellanox MLNX-OS™ User Manual for SX1018HP Ethernet Managed Blade Switch Rev 1.6.2 Software Ver. 3.3.1000 www.mellanox.com...
Page 2
Rev 1.6.2 NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS.
Page 5
Rev 1.6.2 5.8.1 Priority Flow Control and Link Level Flow Control ....70 5.8.2 Enhanced Transmission Selection (ETS)......72 5.9 Access Control List .
Rev 1.6.2 About this Manual This manual provides general information concerning the scope and organization of this User’s Manual. Intended Audience This manual is intended for network administrators who are responsible for configuring and managing Mellanox Technologies’ SwitchX based Switch Platforms. Related Documentation The following table lists the documents referenced in this User’s Manual.
Rev 1.6.2 Glossary Table 3 - Glossary Authentication, Authorization, and Accounting. Authentication - verifies user credentials (username and password). Authorization - grants or refuses privileges to a user/client for accessing spe- cific services. Accounting - tracks network resources consumption by users. Address Resolution Protocol.
Page 9
Rev 1.6.2 Table 3 - Glossary MTU (Maximum Transfer Unit) The maximum size of a packet payload (not including headers) that can be sent /received from a port Network Adapter A hardware device that allows for communication between computers in a network PFC/FC Priority Based Flow Control applies pause functionality to traffic classes OR...
Rev 1.6.2 Introduction Mellanox Operating System (MLNX-OS) enables the management and configuration of Mella- nox Technologies’ SwitchX silicon based switch platforms. MLNX-OS supports the Virtual Pro- tocol Interconnect (VPI) technology which enables it to be used for both Ethernet and InfiniBand technology providing the user with greater flexibility.
Page 11
Rev 1.6.2 Table 5 - Ethernet Features Feature Description Ethernet support • 48K Unicast MAC addresses • VLAN (802.1Q) - 4K • LAG/LACP (802.3ad), 16 links per LAG (36 LAGs) • Rapid Spanning Tree (802.1w) • Flow control (802.3x) • IGMP snooping v1,2 •...
Rev 1.6.2 Getting Started The procedures described in this chapter assume that you have already installed and powered on your switch according to the instructions in the Hardware Installation Guide, which was shipped with the product. Configuring the Switch for the First Time - Installation Wizard Connect to the HP Chassis Manager and run “connect interconnect <slot number 1-7>”.
Page 13
Rev 1.6.2 Table 7 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 2 of 3) Wizard Session Display (Example) Comments Step 2: Use DHCP on mgmt0 interface? [yes] Perform this step to obtain an IP address for the switch.
Page 14
Rev 1.6.2 Table 7 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 3 of 3) Wizard Session Display (Example) Comments You have entered the following information: The wizard displays a summary of your choices and then asks you to confirm the 1.
Page 15
Rev 1.6.2 Table 8 - Configuration Wizard Session - IP Zeroconf Configuration Wizard Session Display - IP Zeroconf Configuration (Example) Mellanox configuration wizard Do you want to use the wizard for initial configuration? y Step 1: Hostname? [switch-112126] Step 2: Use DHCP on mgmt0 interface? [no] Step 3: Use zeroconf on mgmt0 interface? [no] yes Step 4: Default gateway? [192.168.10.1] Step 5: Primary DNS server?
Page 16
Rev 1.6.2 Table 9 - Configuration Wizard Session - Static IP Configuration Wizard Session Display - Static IP Configuration (Example) Mellanox configuration wizard Do you want to use the wizard for initial configuration? y Step 1: Hostname? [switch-112126] Step 2: Use DHCP on mgmt0 interface? [yes] n Step 3: Use zeroconf on mgmt0 interface? [no] Step 4: Primary IP address? 192.168.10.4 Mask length may not be zero if address is not zero (interface mgmt0)
Rev 1.6.2 Check the mgmt0 interface configuration before attempting a remote (for example, SSH) con- Step 5. nection to the switch. Specifically, verify the existence of an IP address. switch # show interfaces mgmt0 Interface mgmt0 state Admin up: Link up: IP address: 169.254.15.134 Netmask:...
Rev 1.6.2 Start a remote secured shell (SSH) to the switch using the command “ssh -l <username> Step 2. <switch ip address>.” rem_mach1 > ssh -l <username> <ip address> Mellanox MLNX-OS Switch Management Last login: Thu Apr 28 11:24:13 2011 from 192.168.10.1 Mellanox Switch switch >...
Page 19
Rev 1.6.2 Figure 2: MLNX-OS Login Window After you log in to MLNX-OS, a (default) status summary window will be displayed containing the following information: Figure 3: Display After Login Mellanox Technologies...
Rev 1.6.2 Licenses MLNX-OS software package can be extended with premium features. Installing a license allows you to access the specified premium features. This section is relevant only to switch systems with an internal management capability. The following licenses are offered with MLNX-OS software: Table 10 - MLNX-OS licenses Valid on product Description...
Rev 1.6.2 Display the installed license(s) using the following command. Step 3. switch (config) # show licenses License 1: <license key> Feature: EFM_SX Valid: yes Active: yes switch (config) # Make sure that the “Valid” and “Active” fields both indicate “yes”. Save the configuration to complete the license installation.
Page 22
Rev 1.6.2 Enter your license key(s) in the text box. If you have more than one license, please enter each Step 3. license in a separate line. Click “Add Licenses” after entering the last license key to install them. If you wish to add another license key in the future, you can simply enter it in the text box and click “Add Licenses”...
Rev 1.6.2 Figure 6: Installed License Save the configuration to complete the license installation. Step 4. If you do not save the installation session, you will lose the installed licenses at the next system boot. 2.4.3 Retrieving Your Lost License Key If you have lost the MLNX-OS license key, contact your authorized Mellanox reseller and pro- vide the switch’s chassis serial number.
Page 24
Rev 1.6.2 Send your Mellanox reseller the following information to obtain the license key: Step 3. • The chassis serial number • The type of license you need to retrieve. Refer to “MLNX-OS licenses” on page 20. Once you receive the license key, you can install the license as described in the sections above. Step 4.
Rev 1.6.2 User Interfaces Command Line Interface (CLI) MLNX-OS is equipped with an industry-standard CLI. The CLI is accessed through SSH or Tel- net sessions, or directly via the console port on the front panel (if it exists). Refer to the MLNX-OS Command Reference Guide for complete set of commands, syntax and examples.
Rev 1.6.2 3.1.2 Syntax Conventions To help you identify the parts of a CLI command, this section explains conventions of presenting the syntax of commands. Table 12 - Syntax Conventions Syntax Convention Description Example < > Angled brackets Indicate a value/variable that <1...65535>...
Rev 1.6.2 slogin Log into another system securely using ssh switch Configure switch on system telnet Log into another system using telnet terminal Set terminal parameters traceroute Trace the route packets take to a destination switch-11a596 [standalone: master] > If you type a legal string and then you press “?” without a space character before it, then you will either get a description of the command that you have typed so far or the possible command/ parameter completions.
Rev 1.6.2 Commands entered do not print any response and simply show the command prompt after you press <Enter>. If an error is encountered in executing a command, the response will begin with “ ”, followed by some text describing the error. 3.1.5 Using the Negation Form Several Config mode commands offer the negation form using the keyword “no”.
Rev 1.6.2 Maximum line size: 8192 Terminal width: 157 columns Terminal length: 60 rows Terminal type: xterm Auto-logout: 15 minutes Paging: enabled Progress tracking: enabled Prefix modes: enabled 3.1.6 Parameter Key This section provides a key to the meaning and format of all of the angle-bracketed parameters in all the commands that are listed in this document.
Rev 1.6.2 Table 13 - Angled Brackets Parameter Description Parameter Description <URL> A normal URL, using any protocol that wget supports, including http, https, ftp, sftp, and tftp; or a pseudo-URL specifying an scp file transfer. The scp pseudo- URL format is scp://username:password@hostname/path/filename. Note that the path is an absolute path.
Rev 1.6.2 Figure 7: WebUI 3.2.1 Setup Menu The Setup menu makes available the following submenus (listed in order of appearance from top to bottom): Table 14 - Setup Submenus Submenu Title Description Interfaces Used to obtain the status of, configure, or disable interfaces to the InfiniBand fabric.
Rev 1.6.2 Table 14 - Setup Submenus Submenu Title Description Used to add static and dynamic ARP entries, and to clear the dynamic ARP cache. IPSec Used to configure IPSec feature. Neighbors Used to display IPv6 neighbor discovery protocol. Virtual Switch Mgmt Used to set the system profile.
Rev 1.6.2 Table 15 - System Submenus Submenu Title Description Reboot Used to reboot the system. Make sure that you save your configuration prior to clicking reboot. 3.2.3 Security Menu The Security menu makes available the following sub-menus (listed in order of appearance from top to bottom): Table 16 - Security Submenus Submenu Title...
Rev 1.6.2 3.2.5 Status Menu The Status menu makes available the following sub-menus (listed in order of appearance from top to bottom): Table 18 - Status Submenus Submenu Title Description Summary Displays general information about the switch system and the MLNX-OS image, including: current date and time, hostname, uptime of system, system memory, CPU load averages, etc.
Rev 1.6.2 The IB SM Mgmt menu makes available the following sub-menus (listed in order of appearance from top to bottom): Table 19 - IB SM Mgmt Submenus Submenu Title Description Summary Displays the local Subnet Manager (SM) status (running time, failures, etc). Base SM Used to manage basic SM configuration (enabling SM, priority level, and restor- ing initial configuration).
Rev 1.6.2 Table 20 - Fabric Inspctr Submenus Submenu Title Description IB Nodes Displays information about InfiniBand nodes in the fabric. It is possible to filter display by the type of InfiniBand node (HCA adapter, switch, etc). IB Ports Displays all active InfiniBand ports in the fabric. It is possible to filter display by the type of InfiniBand port (HCA port, switch port, switch management port, etc), by the port rate (speed or width), by the Subnet Manager status on the node, by node traffic, etc.
Rev 1.6.2 System Management Management Interface 4.1.1 Configuring Management Interfaces With Static IP Addresses If your switch system was set during initialization to obtain dynamic IP addresses through DHCP and you wish to change to static assignments, perform the following steps: Change to Config mode.
Rev 1.6.2 For all other systems (and software versions) DHCP is disabled by default. If a user connects through SSH, runs the wizard and turns off DHCP, the connection is immediately terminated as the management interface loses its IP address. <localhost># ssh admin@192.168.10.101 Mellanox MLNX-OS Switch Management Password:...
Page 39
Rev 1.6.2 SX_PPC_M460EX 3.0.0000-dev-HA 2011-04-10 12:02:49 ppc Partition 2: SX_PPC_M460EX 3.0.0000-dev-HA 2011-04-10 12:02:49 ppc Last boot partition: 1 Next boot partition: 1 Boot manager password is set. No image install currently in progress. Require trusted signature in image being installed: yes switch (config) # Delete the old image that is listed under prior to...
Page 40
Rev 1.6.2 No boot manager password is set. switch (config) # Install the new image. Step 6. switch (config) # image install <image_name> Step 1 of 4: Verify Image 100.0% [#############################################################] Step 2 of 4: Uncompress Image 100.0% [#############################################################] Step 3 of 4: Create Filesystems 100.0% [#############################################################] Step 4 of 4: Extract Image 100.0% [#############################################################]...
Rev 1.6.2 After software reboot, the software upgrade will also automatically upgrade the firm- ware version. On SX65XX systems with dual management, the software must be upgraded on both the Master and the Slave units. In order to upgrade the system on dual management system refer to Section 4.2.1, “Upgrading MLNX-OS Software,”...
Rev 1.6.2 Require trusted signature in image being installed: yes switch (config) # Delete the unused images. Run: Step 3. switch config) # image delete image-PPC_M460EX-SX_3.0.1224.img switch (config) # For further information, please refer to Mellanox MLNX-OS Command Reference Guide. 4.2.3 Downgrading MLNX-OS Software Prior to downgrading software, please make sure the following prerequisites are met:...
Page 43
Rev 1.6.2 4.2.3.2 Downgrading Image The procedure below assumes that booting and running is done from Partition 1 and the downgrade procedure is performed on Partition 2. Log in as admin. Step 1. Enter config mode. Run: Step 2. switch > enable switch # configure terminal Show all image files on the system.
Page 44
Rev 1.6.2 Partition 2: <downgrade version> 2010-09-19 16:52:50 Last boot partition: 1 Next boot partition: 2 No boot manager password is set. switch (config) # Set the boot location to be the other partition (next). Run: Step 6. switch (config) # image boot next There are two installed images on the system.
Rev 1.6.2 Set the boot partition. Run: Step 2. switch (config)# image boot next Save the configuration. Run: Step 3. switch (config)# configuration write Reload the system. Run: Step 4. switch (config)# reload 4.2.4 Upgrading System Firmware Each MLNX-OS software package version has a default switch firmware version. When you update the MLNX-OS software to a new version, an automatic firmware update process will be attempted by MLNX-OS.
Rev 1.6.2 4.2.4.2.1Default Firmware Change on Standalone Systems Import the firmware image (.tgz file). Run: Step 1. switch (config) # image fetch switch (config) # image default-chip-fw fw-SX-rel-9_1_2090.tgz Default Firmware 9.1.2090 updated. Please save configuration and reboot for new FW to take effect.
Rev 1.6.2 To display the available configuration files and the active file, run: switch (config) # show configuration files initial myconf (active) switch (config) # For further information, please refer to Mellanox MLNX-OS Command Reference Guide. 4.3.2 Loading a Configuration File By default, or after a system reset, the system loads the default “initial”...
Rev 1.6.2 Set the minimum severity of the log level to . Run: Step 3. info switch (config) # logging <IP address> trap info Override the log levels on a per-class basis. Run: Step 4. switch (config) # logging <IP address> trap override class <class name> For further information, please refer to Mellanox MLNX-OS Command Reference Guide.
Rev 1.6.2 Diagnostics Retrieving Return Codes when Executing Remote CLI Commands 4.6.1 through SSH To stop the CLI and set the system to send return errors if some commands fail, perform the following: Connect to the system from the host SSH. Step 1.
Page 50
Rev 1.6.2 4.7.1.1 RADIUS RADIUS (Remote Authentication Dial-In User Service), widely used in network environments, is a client/server protocol and software that enables remote access servers to communicate with a central server to authenticate dial-in users and authorize their access to the requested system or service.
Rev 1.6.2 For information on the LDAP commands, please refer to Mellanox MLNX-OS Command Refer- ence Guide. 4.7.2 Secure Shell (SSH) 4.7.2.1 Adding a Host and Providing an SSH Key To add entries to the global known-hosts configuration file and its SSH value, perform the following steps: Change to Config mode Run: Step 1.
Rev 1.6.2 Network Management Interfaces 4.8.1 SNMP Simple Network Management Protocol (SNMP), is a network protocol for the management of a network and the monitoring of network devices and their functions. SNMP supports asynchro- nous event (trap) notifications and queries. MLNX-OS supports: •...
Page 53
Rev 1.6.2 Table 24 - Standard MIBs - Structure, Management interface and General SNMP Standard Comments IP-MIB RFC 4293 Management interface TCP-MIB RFC 4022 Management interface UDP-MIB RFC 4113 Management interface IP-FORWARD-MIB RFC 4292 Management interface HOST-RESOURCES-MIB, RFC 2790 Management interface HOST-RESOURCES-TYPE Table 25 - Standard MIBs - Chassis and Switch Standard...
Page 54
Rev 1.6.2 Table 25 - Standard MIBs - Chassis and Switch Standard Comments LLDP-MIB 802.1AB-2005 This MIB is not relevant to InfiniBand. 4.8.1.2 Private MIB Table 26 - Private MIBs Supported Comments MELLANOX-SMI-MIB Mellanox Private MIB main structure (no objects) MELLANOX-PRODUCTS-MIB List of OID - per managed system (sysObjID) MELLANOX-IF-VPI-MIB...
Page 55
Rev 1.6.2 4.8.1.5 Configuring an SNMPv3 User To configure SNMP V3 user: Configure the user using the command: Step 1. switch (config) # snmp-server user [role] v3 prompt auth <hash type> priv <privacy type> where • user role - admin •...
Page 57
Rev 1.6.2 SSH, and then it uses the existing CLI authentication mechanism. XML Gateway - Management information base. For further information please contact Mellanox support. Mellanox Technologies...
Rev 1.6.2 Ethernet Switching Interface Interface Ethernet have the following physical set of configurable parameters • Admin state - enabling or disabling the interface. • Flow control - admin state per direction (send or receive) • MTU (Maximum Transmission Unit) - (1518-9216 bytes) •...
Page 59
Rev 1.6.2 A split-4 operation results in blocking a 40G port in addition to the one being split. A set of hard- ware restrictions determine which of the ports can be split. Specific ports can be split by using a QSFP 1X4 breakout cable to split one 40 Gb/s port into 4 lanes (4 SFP+ connectors).
Page 60
Rev 1.6.2 Table 27 - Port Splitting Options Can be split Turns off port Can be split Port # to 4 to 2 — — — — — — — — — — — — — — — — — —...
Rev 1.6.2 5.1.1.2 Returning to the Non-Split Mode To return to the non-split mode: Log in as admin. Step 1. Enter config mode. Run: Step 2. switch > enable switch # configure terminal Shut down all of the split ports. Run: Step 3.
Rev 1.6.2 Each port can have a switch mode of either: • Access - Access port is a port connected to a host. It can accept only untagged frames, and assigns them a default configured VLAN (Port VLAN ID). On egress, traffic sent from the access port is untagged.
Rev 1.6.2 Spanning Tree 5.5.1 Rapid Spanning Tree Protocol The operation of Rapid Spanning Tree Protocol (RSTP) provides for rapid recovery of connectiv- ity following the failure of a bridge/bridge port or a LAN. The RSTP component avoids this delay by calculating an alternate root port, and immediately switching over to the alternate port if the root port becomes unavailable.
Rev 1.6.2 • Per interface cost: Applies a cost to the interface spanning-tree cost [ 1 - 200000000] 5.5.3 Disabling Spanning Tree Protocol STP is used to recover connectivity following the failure of a bridge/bridge port or a LAN, and is enabled by default.
Rev 1.6.2 5.6.2 Defining a Multicast Router Port on a VLAN You can define a Multicast Router (MRouter) port on a VLAN in one of the following methods: To change the Interface Switchport to Trunk: Log in as admin. Step 1.
Rev 1.6.2 Show LLDP remote information. Run: Step 6. switch (config)# show lldp ethernet 1/1 remote Ethernet 1/1 Remote Index: 1 Remote chassis id: 00:11:22:33:44:55 ; chassis id subtype: mac Remote port-id: ethenret 1/2; port id subtype: local Remote port description: ethernet 1/2 Remote system name: remote-system Remote system description: remote-system-description Remote system capabilities supported: B ;...
Page 71
Rev 1.6.2 Log in as admin. Step 1. Enter config mode. Run: Step 2. switch > enable switch # configure terminal Enable PFC globally on the switch. Run: Step 3. switch (config) # dcb priority-flow-control enable This action might cause traffic loss while shutting down a port with priority-flow-con- trol mode on Type 'yes' to confirm enable pfc globally: yes ...
Rev 1.6.2 5.8.2 Enhanced Transmission Selection (ETS) Enhanced Transmission Selection (ETS) provides a common management framework for assign- ment of bandwidth to traffic classes, for weighted round robin (WRR) scheduling. If a traffic class does not use all the bandwidth allocated to it, other traffic classes can use that available bandwidth.
Rev 1.6.2 traffic on TC-0, TC-1, TC-2 while reducing the amount of traffic sent on TC-3, run the com- mand dcb ets tc bandwidth. switch (config) # dcb ets tc bandwidth 30 30 30 10 # show dcb ets ETS enabled Bandwidth -------------------------- Number of Traffic Class: 4...
Rev 1.6.2 5.9.1 Configuring Access Control List Access Control List (ACL) is configured by the user and is applied to a port once the ACL search engine matches search criteria with a received packet. To configure ACL: Log in as admin. Step 1.
Rev 1.6.2 IP Routing General 6.1.1 Interface VLAN Interface VLAN is an IPv4 interface created per subnet over a specific 802.1Q VLAN ID. If two hosts from two different subnets need to communicate (via the IP layer), the network administra- tor needs to configure two interface VLANs, one for each of the subnets.
Rev 1.6.2 The load balancing function of the ECMP is configured globally on the system. Hash algorithm can be symmetric or asymmetric. In symmetric hash functions bidirectional flows between routes will follow the same path, while in asymmetric hash functions, bidirec- tional traffic can follow different paths in both directions.
Need help?
Do you have a question about the Mellanox SX1018 and is the answer not in the manual?
Questions and answers