Related Manuals for Mellanox Technologies ConnectX 3 Pro
Summary of Contents for Mellanox Technologies ConnectX 3 Pro
- Page 1 ConnectX®-3 Pro 10Gb/s Ethernet Single and Dual SFP+ Port Network Interface Card User Manual for Open Compute Project P/N: MCX341A-XCPN, MCX342A-XCPN, MCX341A-XCQN, MCX342A-XCQN Rev 1.1 www.mellanox.com...
- Page 2 KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY.
-
Page 3: Table Of Contents
4.2.8 Uninstalling MLNX_WinOF ..........31 4.2.8.1 Attended Uninstall 4.2.8.2 Unattended Uninstall Mellanox Technologies... - Page 4 Appendix E Sicherheitshinweise (Warnings in German)....... 54 Appendix F Advertencias de seguridad para la instalación (Warnings in Spanish) ..56 Mellanox Technologies...
-
Page 5: List Of Figures
Figure 12: NC-SI Interface Pinout ..........49 Mellanox Technologies... -
Page 6: List Of Tables
I2C-compatible Connector Pinout ......... . 49 Mellanox Technologies... -
Page 7: Revision History
ConnectX®-3 Pro 10Gb/s Ethernet Single and Dual SFP+ Port Network Interface Card User Manual for OCP Rev 1.1 Revision History This document was printed on January 17, 2016. Table 1 - Revision History Table Date Comments/Changes January 2016 Updated Section 3.4, “Card Installation Instructions,” on page 15. January 2015 First Release Mellanox Technologies... -
Page 8: About This Manual
Rev 1.1 About this Manual This User Manual describes Mellanox Technologies ConnectX®-3 Pro 10 Gigabit Ethernet Single and Dual SFP+ Port PCI Express x8 network interface cards. It provides details as to the interfaces of the board, specifications, required software and firmware for operating the board, and relevant documentation. - Page 9 Rev 1.1 Online Resources • Mellanox Technologies web pages: http://www.mellanox.com • Mellanox Technologies Firmware download web page: http://www.mellanox.com => Support => Download Center Document Conventions When discussing memory sizes, MB and MBytes are used in this document to mean size in mega bytes.
-
Page 10: Chapter 1 Introduction
Introduction Introduction This is the User Guide for Mellanox Technologies Ethernet network interface cards based on the ConnectX®-3 Pro EN integrated circuit device for Open Compute Project (OCP). These cards' connectivity provide the highest performing and most flexible interconnect solution... -
Page 11: Features And Benefits
ConnectX-3 Pro SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server. SR-IOV I/O virtualization with ConnectX-3 Pro gives data center managers better server utilization while reducing cost, power, and cable complexity. Mellanox Technologies... -
Page 12: Operating Systems/Distributions
• VMware ESX Server 3.5, vSphere 4.0/4.1 Connectivity • Interoperable with 10GbE switches • Passive copper cable with ESD protection • Powered connectors for optical and active cable support • QSFP to SFP+ connectivity through QSA module • Passive copper cable with ESD protection Mellanox Technologies... -
Page 13: Chapter 2 Interfaces
• 2.5, 5.0, or 8.0GT/s link rate x8 • Auto-negotiates to x8, x4, x2, or x1 • Support for MSI/MSI-X mechanisms C-compatible Interface A three hole footprint for I C harness is provided as the I2C-compatible interface. See Figure Figure 1: Harness Mellanox Technologies... -
Page 14: Led Interface
When the main CPU is powered-down or the adapter network controller is not running, messages that target the adapter's NC-SI are forwarded directly by the adapter to the BMC using the NC-SI interface without being copied to the host memory. Appendix B.3, “NC-SI Interface Pinout,” on page Mellanox Technologies... -
Page 15: Chapter 3 Hardware Installation
Please refrain from touching the cage directly during installation. The cards require a PCI Express x4 or x8 Mezzanine connector. Please consult the host machine documentation for instructions on how to install a PCI Express Mezzanine connector. Mellanox Technologies... -
Page 16: Cables And Modules
6. In the Value display box, check the fields VEN and DEV (fields are separated by ‘&’). In the display example above, notice the sub-string “PCI\VEN_15B3&DEV_1003”: VEN is equal to 0x15B3 – this is the Vendor ID of Mellanox Technologies; and DEV is equal to 1003 – this is a valid Mellanox Technologies PCI Device ID. -
Page 17: On Linux
Rev 1.1 If the PCI device does not have a Mellanox adapter ID, return to Step 2 to check another device. The list of Mellanox Technologies PCI Device IDs can be found in the PCI ID reposi- tory at http://pci-ids.ucw.cz/read/PC/15b3. 3.6.2 On Linux Get the device location on the PCI bus by running lspci and locating lines with the string “Mella-... -
Page 18: Chapter 4 Driver Installation
Please download the current driver package from http://www.mellanox.com => Products => Software => Ethernet Driver => Linux Driver => Download. Install Driver Step 2. Run the following commands to install the driver: #> tar xzvf mlnx_en-1.5.10.tgz file #> cd mlnx_en-1.5.10 #> ./install.sh Mellanox Technologies... -
Page 19: Loading The Driver
#> modprobe mlx4_en The result is a new net-device appearing in 'ifconfig -a' output. 4.1.4 Unloading the Driver To unload the Ethernet driver run: #> modprobe mlx4_en 4.1.5 Uninstalling the Driver To uninstall the mlnx_en driver run: #> /sbin/mlnx_en_uninstall.sh Mellanox Technologies... -
Page 20: Windows Driver
Installing the incorrect .exe file is prohibited. If you do so, an error message will be dis- played. For example, if you try to install a 64-bit .exe on a 32-bit machine, the wizard will dis- play the following (or a similar) error message: Mellanox Technologies... -
Page 21: Extracting Files Without Running Installation
Open a CMD console (Click Start-->Run and enter CMD). Step 1. Enter the following command: Step 2. MLNX_VPI_WinOF-<version>_All_<OS>_<arch>.exe /a Click Next to create a server image. Step 3. Click Change and specify the location in which the files are extracted to. Step 4. Mellanox Technologies... -
Page 22: Installing Mlnx_Winof
Click Install to extract this folder, or click Change to install to a different folder. Step 5. To complete the extraction, click Finish. Step 6. 4.2.4 Installing MLNX_WinOF This section provides instructions for two types of installation procedures: • “Attended Installation” An installation procedure that requires frequent user intervention. • “Unattended Installation” Mellanox Technologies... -
Page 23: 4.2.4.1 Attended Installation
CMD console: MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /v"/l*vx [LogFile]" If you do not want to upgrade your firmware version, run the following command: MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /v" MT_SKIPFWUPGRD=1" For further help, please run: MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /v" /h" Click Next in the Welcome screen. Step 2. Mellanox Technologies... - Page 24 Select the target folder for the installation. Step 4. The firmware upgrade screen will be displayed in the following cases: Step 5. • If the user has an OEM card, in this case the firmware will not be updated. Mellanox Technologies...
- Page 25 WinOF Installation Guide 4.40, the firmware will be updated accordingly. However, if the user has both OEM card and Mellanox card, only Mellanox card will be updated. Configure your system for maximum performance by checking the maximum performance Step 6. box. Mellanox Technologies...
- Page 26 Band environment. • SDK - contains the libraries and DLLs for developing InfiniBand application over IBAL. • Documentation: contains the User Manual and Installation Guide. • ND FLTR DLLs: contains the files for standalone installation of the mlx4nd provider. Mellanox Technologies...
- Page 27 ConnectX®-3 Pro 10Gb/s Ethernet Single and Dual SFP+ Port Network Interface Card User Manual for OCP Rev 1.1 Click Install to start the installation. Mellanox Technologies...
-
Page 28: 4.2.4.2 Unattended Installation
4.2.4.2 Unattended Installation The following is an example of a MLNX_WinOF_win8 x64 unattended installation session. Open the CMD console (click Start > Run and enter ‘cmd’) Step 1. Install the driver. Run: Step 2. > MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /S /v"/qn" Mellanox Technologies... -
Page 29: Upgrading Mlnx_Winof
Upon installation completion, you can verify the successful addition of the network card(s) through the Device Manager. To see the Mellanox network adapter device, and the Ethernet or IPoIB network device (depending on the used card) for each port, display the Device Manager and expand “System devices” or “Network adapters”. Mellanox Technologies... -
Page 30: Opensm Activation
To run OpenSM as a service, assuming the package was installed in the default path, use: sc create OpenSM1 binPath= "c:\Program Files\Mellanox\MLNX_VPI\IB\Tools\ opensm.exe --service" start=auto" To start the service, run: sc start opensm For further information, please refer to the “OpenSM - Subnet Manager” chapter in the User Manual. Mellanox Technologies... -
Page 31: Uninstalling Mlnx_Winof
1. Click Start-> Control Panel-> Programs and Features-> MLNX_VPI-> Uninstall. (NOTE: This requires elevated administrator privileges.) 2. Double click the .exe and follow the instructions of the install wizard. 3. Click Start-> All Programs-> Mellanox Technologies-> MLNX_WinOF-> Uninstall MLNX- _WinOF. 4.2.8.2 Unattended Uninstall ... -
Page 32: Vmware Driver
For VMware download and install the latest Mellanox OFED Driver for VMware® ESXi Server- software package available via the Mellanox web site at: http://www.mellanox.com => Products => Software => Ethernet Drivers => VMware Drivers => Download. Follow the installation instructions included in the download package (also available from the download page). Mellanox Technologies... -
Page 33: Installing And Running The Vbi Driver On Esxi-5.X
Run: #> esxcli software vib list #> esxcli software vib remove –n net-mlx4-en b. Install the mlx4_en driver offline_bundle package. Run: #> esxcli software vib install –d <path>/mlx4_en-mlnx-1.6.1.2-offline_bundle-471530.zip c. Reboot ESXi server. (The driver will be loaded automatically). Mellanox Technologies... -
Page 34: Removing The Vib/Offline_Bundle Driver
FlexBoot supports remote Boot over Ethernet. This technology is based on the Preboot Execution Environment (PXE) standard specification, and FlexBoot software is based on the open source iPXE project (see www.ipxe.org). For more information go to http://www.mellanox.com => Products => Software => Ethernet Drivers => Download. Mellanox Technologies... -
Page 35: Chapter 5 Updating Card Firmware
Enter your card PSID to display the latest firmware file. The file name of the binary is composed by combin- ing the firmware name, the firmware release version, and the card part number. Note: Please contact Mellanox System Support if you cannot find the firmware binary for your card. Mellanox Technologies... -
Page 36: Figure 3: Support Download Assistant
> flint -d /dev/mst/mt4099_pci_cr0 -i <binary image> burn a. Reboot the computer. b. Enter: mst start. c. Verify that the card firmware was updated successfully. > flint -d /dev/mst/mt4099_pci_cr0 Image type: ConnectX FW Version: 2.9.4100 Device ID: 4099 Mellanox Technologies... -
Page 37: Chapter 6 Troubleshooting
• Verify that your switch and adapter port are compatible • Check that the latest driver is loaded Link light is on, but • Check that both the adapter and its link are set to the same speed with no and duplex settings communication established Mellanox Technologies... -
Page 38: Linux
Mellanox Firmware Tool (MFT) Once installed, run: mst start mst status flint –d <mst_device> q ibstat Ports Information lbv_devinfo Firmware Version To download the latest firmware version refer to Upgrade http://www.mellanox.com/supportdownloader /var/log/messages Collect Log File dmesg > system.logF Mellanox Technologies... -
Page 39: Windows
–d <mst_device> –i <firmware_bin_file> b • Event log viewer • MST device logs: • mst start Collect log file • mst status • C:\Users\Administrator> flint –d <mst_device> dc > dump_configura- tion.log • C:\Users\Administrator> mstdump <mst_device> dc > mstdump.log Mellanox Technologies... -
Page 40: Chapter 7 Specifications
IEC 60068-2- 64, 29, 32 RoHS: RoHS-R6 Please refer to www.mellanox.com => Products => Cables and Transceivers Cable Support a. For both operational and non-operational states b. Air flow is measured ~1” from the Mezz from the cooling air inlet side. Mellanox Technologies... -
Page 41: Mcx342A-Xc[P/Q]N Specifications
IEC 60068-2- 64, 29, 32 RoHS: RoHS-R6 Please refer to www.mellanox.com => Products => Cables and Transceivers Cable Support a. For both operational and non-operational states b. Air flow is measured ~1” from the Mezz from the cooling air inlet side. Mellanox Technologies... -
Page 42: Board Mechanical Drawing And Dimensions
Rev 1.1 Specifications Board Mechanical Drawing and Dimensions All dimensions are in millimeters. All the mechanical tolerances are +/- 0.1mm Figure 4: Mechanical Drawing of the Single-port MCX341A Mezzanine Card Mellanox Technologies... -
Page 43: Adapter Led Operation
Physical link - Yellow Other link status is present P0-LED1 Logical Link/ Activity - Green P1-LED0 Physical link - Green 10G link is present Physical link - Yellow Other link status is present P1-LED1 Logical Link/ Activity - Green Mellanox Technologies... -
Page 44: Appendix A Finding The Mac And Serial Number On The Card
Note: The revision indicated on the labels in the following figures do not necessarily represent the latest revision of the card. Card revision changes are communicated via Product Change Notification (PCN) documents that are available via card suppliers. Figure 6: MCX341A-XCPN Board Label (Example only - see Note) Mellanox Technologies... -
Page 45: Figure 7: Mcx341A-Xcqn Board Label (Example Only - See Note)
ConnectX®-3 Pro 10Gb/s Ethernet Single and Dual SFP+ Port Network Interface Card User Manual for OCP Rev 1.1 Figure 7: MCX341A-XCQN Board Label (Example only - see Note) Figure 8: MCX342A-XCPN Board Label (Example only - see Note) Mellanox Technologies... -
Page 46: Retrieving Card Guid
GUID via device firmware (e.g., using ibstat). Mlxburn/flint returns 0xffff as GUID while the utilities return a value derived from the MAC address. For all driver/firmware/software purposes, the latter value should be used. Please see the below example. Mellanox Technologies... - Page 47 -d /dev/mst/mt4103_pci_cr0 -qq q -W- Running quick query - Skipping full image integrity checks. Image type: ConnectX FW Version: 2.30.5000 Device ID: 4103 Description: Node Port1 Port2 Sys image GUIDs: 0002c90300e8eef0 0002c90300e8eef1 0002c90300e8eef2 0002c90300e8eef3 MACs: 0002c9e8eef0 0002c9e8eef1 VSD: PSID: MT_1200111023 Mellanox Technologies...
-
Page 48: Appendix B Interface Connectors Pinout
Appendix B: Interface Connectors Pinout PCI Express x8 Connector Pinout The cards use a standard PCI Express x8 connector and the PCI Express x8 standard pinout according to the PCI Express 3.0 specification. Figure 10: PCIe Connector Pinout Mellanox Technologies... -
Page 49: I2C-Compatible Connector Pinout
A three hole footprint for I2C Harness is provided as the I2C-compatible interface. Figure 11: Compatible Connector Plug and Pinout Table 11 - I2C-compatible Connector Pinout Connector Pin Number Signal Name NC-SI Interface Pinout Figure 12: NC-SI Interface Pinout Mellanox Technologies... -
Page 50: Appendix C Safety Warnings
This equipment should be installed in compliance with local and national electrical codes. 6. Hazardous Radiation Exposure Caution – Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure. Mellanox Technologies... - Page 51 ConnectX®-3 Pro 10Gb/s Ethernet Single and Dual SFP+ Port Network Interface Card User Manual for OCP Rev 1.1 CLASS 1 LASER PRODUCT and reference to the most recent laser standards: IEC 60 825-1:1993 + A1:1997 + A2:2001 and EN 60825-1:1994+A1:1996+ A2:2001 Mellanox Technologies...
-
Page 52: Appendix D Avertissements De Sécurité D'installation (Warnings In French)
L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur. 6. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies... - Page 53 PRODUIT LASER DE CLASSE 1 » et références aux normes laser les plus récentes CEI 60 825-1:1993 + A1:1997 + A2:2001 et NE 60825- 1:1994+A1:1996+ A2:2001 Mellanox Technologies...
-
Page 54: Appendix E Sicherheitshinweise (Warnings In German)
5. Geräteentsorgung Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestim- mungen erfolgen. 6. Regionale und nationale elektrische Bestimmungen Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bes- timmungen installiert werden. Mellanox Technologies... - Page 55 Achtung – Nutzung von Steuerungen oder Einstellungen oder Ausführung von Prozeduren, die hier nicht spezifiziert sind, kann zu gefährlichem Strahlenkon- takt führen.. Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards : ICE 60 825-1:1993 + A1:1997 + A2:2001 und EN 60825-1:1994+A1:1996+ A2:2001 Mellanox Technologies...
-
Page 56: Appendix F Advertencias De Seguridad Para La Instalación (Warnings In Spanish)
5. Eliminación de equipos La eliminación definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales. 6. Códigos eléctricos locales y nacionales Este equipo se debe instalar conforme a los códigos eléctricos locales y nacionales. Mellanox Technologies... - Page 57 Precaución: el uso de controles o ajustes o la realización de procedimientos distintos de los que aquí se especifican podrían causar exposición a niveles de radiación peligrosos. PRODUCTO LÁSER DE CLASE 1 y referencia a las normas de láser más recientes: IEC 60825-1:2007/03 y EN 60825-1:2007 Mellanox Technologies...
Need help?
Do you have a question about the ConnectX 3 Pro and is the answer not in the manual?
Questions and answers