Subscribe to Our Youtube Channel
Related Manuals for Mellanox Technologies ConnectX MCX454A-FCAT
Summary of Contents for Mellanox Technologies ConnectX MCX454A-FCAT
- Page 1 ConnectX®-4 VPI Single and Dual QSFP28 Adapter Card User Manual P/N: MCX453A-FCAT, MCX454A-FCAT, MCX455A-FCAT, MCX456A-FCAT, MCX455A-ECAT, MCX456A- ECAT Rev 1.9 www.mellanox.com...
- Page 2 KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY.
-
Page 3: Table Of Contents
4.1 Linux Driver ........... . . 24 Mellanox Technologies... - Page 4 7.11 PCI Express x8 orx16 Edge Connector Pinout ......52 Appendix B Finding the GUID/MAC and Serial Number on the Adapter Card ..54 Mellanox Technologies...
- Page 5 Appendix E Sicherheitshinweise (Warnings in German) ..... . . 61 Appendix F Advertencias de seguridad para la instalación (Warnings in Spanish) . 63 Mellanox Technologies...
-
Page 6: List Of Tables
Physical and Logical Link Indications ........46 Mellanox Technologies... -
Page 7: List Of Figures
Figure 16: MCX456A-ECAT Board Label ..........56 Mellanox Technologies... -
Page 8: Revision History
Card on page 54. • Updated the following Board Labels: • MCX453A-FCAT Board Label on page 54. • MCX454A-FCAT Board Label on page 54. • MCX455A-FCAT Board Label on page 55. • MCX456A-FCAT Board Label on page 55. Mellanox Technologies... - Page 9 • MCX455A-FCAT Specifications on page 42. • MCX456A-FCAT Specifications on page 43. • Added Airflow Specifications on page 46. • Added VMware Driver on page 35. • Updated Updating Adapter Card Firmware on page 37. June 2015 First Release Mellanox Technologies...
-
Page 10: About This Manual
Rev 1.9 About this Manual This User Manual describes Mellanox Technologies ConnectX®-4 VPI Single and Dual QSFP28 ports PCI Express x8 and x16 adapter card. It provides details as to the interfaces of the board, specifications, required software and firmware for operating the board, and relevant docu- mentation. -
Page 11: Technical Support
The Mellanox support downloader contains software, firmware and knowledge database infor- mation for Mellanox products. Access the database from the Mellanox Support web page, http://www.mellanox.com => Support Or use the following link to go directly to the Mellanox Support Download Assistant page, http://www.mellanox.com/supportdownloader/. Mellanox Technologies... -
Page 12: Chapter 1 Introduction
Rev 1.9 Introduction This is the User Guide for Mellanox Technologies VPI adapter cards based on the ConnectX®-4 integrated circuit device. These adapters connectivity provide the highest performing and most flexible interconnect solution for PCI Express Gen3 servers used in Enterprise Data Centers, High-Performance Computing, and Embedded environments. -
Page 13: Table 4: Single And Dual-Port Edr Adapter Cards
Ethernet: 1/10/25/40/50/100 Gb/s Single or Dual-port QSFP28 Network Connector Types PCIe 3.0 x16 8GT/s PCI Express (PCIe) SerDes Speed RoHS MT27704A0-FDCF-EV Adapter IC Part Number MT27708A0-FDCF-EV 4115 for Physical Function (PF) Device ID 4116 for Virtual Function (VF) Mellanox Technologies... -
Page 14: Features And Benefits
TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-4 effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol. Mellanox Technologies... -
Page 15: Operating Systems/Distributions
Table 2, “Documents List,” on page Operating Systems/Distributions • RHEL/CentOS • FreeBSD • OpenFabrics Enterprise Distribution (OFED) Connectivity • Interoperable with InfiniBand or 1/10/25/40/50/56/100 Gb/s Ethernet switches • Passive copper cable with ESD protection • Powered connectors for optical and active cable support Mellanox Technologies... -
Page 16: Chapter 2 Interfaces
• Auto-negotiates to x16, x8, x4, x2, or x1 • Support for MSI/MSI-X mechanisms LED Interface There is a one bi-color I/O LED per port. For LED specifications, please refer to Section 7.8, “Adapter Card LED Operations,” on page Mellanox Technologies... -
Page 17: Chapter 3 Hardware Installation
19. If you need to replace it with the short bracket that is included in the shipping box, please follow the instructions in this section. Due to risk of damaging the EMI gasket, it is not recommended to replace the bracket more than three times. Mellanox Technologies... -
Page 18: Removing The Existing Bracket
2. Screw on the bracket using the screws and washers saved from the bracket removal procedure above. 3. Make sure that the LEDs are aligned onto the bracket holes. 4. Use a torque driver to apply up to 2.9 lbf-in torque on the screws. Mellanox Technologies... -
Page 19: Card Installation Instructions
2. Open the system case. 3. Locate an available PCI Express slot for the adapter card. 4. Applying even pressure at both corners of the card, insert the adapter card into the PCI Express slot until firmly seated. Mellanox Technologies... -
Page 20: Cables And Modules
Insert the connector into the adapter card. Be careful to insert the connector straight into the cage. Do not apply any torque, up or down, to the connector cage in the adapter card. d. Make sure that the connector locks in place. When installing cables make sure that the latches engage. Mellanox Technologies... -
Page 21: Adapter Card Un-Installation Instructions
Please note that the following images are for illustration purposes only. 1. Verify that the system is powered off and unplugged. 2. Wait 30 seconds. 3. To remove the card, disengage the retention mechanisms on the bracket (clips or screws). Mellanox Technologies... -
Page 22: Identify The Card In Your System
2. Expand System Devices and locate your Mellanox ConnectX-4 adapter card. 3. Right click the mouse on your adapter's row and select properties to display the adapter card properties window. 4. Click the Details tab and select Hardware Ids (Windows 2012/R2) from the Properties pull- down menu. Mellanox Technologies... -
Page 23: On Linux
5. In the Value display box, check the fields VEN and DEV (fields are separated by ‘&’). In the display example above, notice the sub-string “PCI\VEN_15B3&DEV_1003”: VEN is equal to 0x15B3 – this is the Vendor ID of Mellanox Technologies; and DEV is equal to 1003 – this is a valid Mellanox Technologies PCI Device ID. -
Page 24: Chapter 4 Driver Installation
Step 1. The following example shows a system with an installed Mellanox HCA: # lspci -v | grep Mellanox 86:00.0 Network controller [0207]: Mellanox Technologies MT27620 Family Subsystem: Mellanox Technologies Device 0014 86:00.1 Network controller [0207]: Mellanox Technologies MT27620 Family Subsystem: Mellanox Technologies Device 0014 Download the ISO image to your host. -
Page 25: Installing Mellanox Ofed
Support (DKMS) framework. Thus, the drivers' compilation will take place on the host during MLNX_OFED installation. Therefore, using is irrelevant on Ubuntu and "mlnx_add_kernel_support.sh" Debian distributions. 1. The firmware will not be updated if you run the install script with the ‘--without-fw-update’ option. Mellanox Technologies... -
Page 26: Installation Procedure
Note that all other Mellanox, OEM, OFED, or Distribution IB packages will be removed. Uninstalling the previous version of MLNX_OFED_LINUX Starting MLNX_OFED_LINUX-x.x.x installation ....Installation finished successfully. Attempting to perform Firmware update... Querying Mellanox devices firmware ... Mellanox Technologies... - Page 27 Most of the Mellanox OFED components can be configured or reconfigured after the installation by modifying the relevant configuration files. See the relevant chapters in this manual for details. The list of the modules that will be loaded automatically upon boot can be found in the /etc/ file. infiniband/openib.conf Mellanox Technologies...
-
Page 28: Installation Results
This can be controlled by setting the variables below in the /etc/infiniband/openibd.conf file. OPENIBD_PRE_START OPENIBD_POST_START OPENIBD_PRE_STOP OPENIBD_POST_STOP Example: OPENIBD_POST_START=/sbin/openibd_post_start.sh 4.1.3.5 Driver Load Upon System Boot Upon system boot, the Mellanox drivers will be loaded automatically. To prevent automatic load of the Mellanox drivers upon system boot: Mellanox Technologies... -
Page 29: Mlnxofedinstall Return Codes
# mount -o ro,loop MLNX_OFED_LINUX-<ver>-<OS label>-<CPU arch>.iso /mnt You can download it from http://www.mellanox.com => Products => Software => Infini- Band Drivers. Download and install Mellanox Technologies GPG-KEY: Step 3. The key can be downloaded via the following link: Mellanox Technologies... -
Page 30: Installing Mlnx_Ofed Using The Yum Tool
# sudo rpm --import RPM-GPG-KEY-Mellanox warning: rpmts_HdrFromFdno: Header V3 DSA/SHA1 Signature, key ID 6224c050: NOKEY Retrieving key from file:///repos/MLNX_OFED/<MLNX_OFED file>/RPM-GPG-KEY-Mellanox Importing GPG key 0x6224C050: Userid: "Mellanox Technologies (Mellanox Technologies - Signing Key v2) <support@mel- lanox.com>" From : /repos/MLNX_OFED/<MLNX_OFED file>/RPM-GPG-KEY-Mellanox Is this ok [y/N]: Check that the key was successfully imported. - Page 31 MLNX_OFED, you can use the mlnx_add_ker- script to build MLNX_OFED for your kernel. nel_support.sh The script will automatically build the matching group RPM packages for your kernel so that you can still install MLNX_OFED via yum. Mellanox Technologies...
-
Page 32: Uninstalling Mellanox Ofed Using The Yum Tool
To update the firmware to the version included in MLNX_OFED package, you can either: • Run # yum install mlnx-fw-updater • Update the firmware to the latest version available on Mellanox Technologies’ Web site as described in Section 4.1.7, “Updating Firmware After Installation,” on page 4.1.5.3 Uninstalling Mellanox OFED using the YUM Tool... -
Page 33: Installing Mlnx_Ofed Using The Apt-Get Tool
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 Check that the key was successfully imported. Step 5. # apt-key list 1024D/A9E4B643 2013-08-11 Mellanox Technologies <support@mellanox.com> 1024g/09FCC269 2013-08-11 Update the apt-get cache. Step 6. # sudo apt-get update 4.1.6.2 Installing MLNX_OFED using the apt-get Tool... -
Page 34: Updating The Device Online
OEM card and now you wish to (manually) update firmware on your adapter card(s), you need to perform the steps below. The following steps are also appropriate in case you wish to burn newer firmware that you have downloaded from Mellanox Technologies’ Web site (http:// www.mellanox.com... -
Page 35: Uefi Secure Boot
UEFI Secure Boot is enabled. To see what keys have been added to the system key ring on the current boot, install the 'keyutils' package and run: #keyctl list %:.system_keyring Mellanox Technologies... -
Page 36: Removing Signature From Kernel Modules
After the signature has been removed, a massage as the below will no longer be presented upon module loading: "Request for unknown module key 'Mellanox Technologies signing key: 61feb074fc7292f958419386ffdd9d5ca999e403' err -11" However, please note that a similar message as the following will still be presented: "my_module: module verification failed: signature and/or required key missing - taint-... -
Page 37: Chapter 5 Updating Adapter Card Firmware
PCI Device Name: 0000:07:00.0 Base MAC: 0000e41d2da206d4 Versions: Current Available 12.12.0780 12.12.1100 Status: Update required Perform FW update? [y/N]: y Device #1: Up to date Device #2: Updating FW ... Done Restart needed for updates to take effect. Log File: /var/log/mlxup/mlxup-yyyymmdd.log Mellanox Technologies... -
Page 38: Chapter 6 Troubleshooting
• Verify that your switch and adapter port are compatible • Check that the latest driver is loaded Link light is on, but • Check that both the adapter and its link are set to the same speed with no and duplex settings communication established Mellanox Technologies... -
Page 39: Linux
Mellanox Firmware Tool (MFT) Once installed, run: mst start mst status flint –d <mst_device> q ibstat Ports Information lbv_devinfo To download the latest firmware version refer to Firmware Version http://www.mellanox.com/supportdownloader Upgrade /var/log/messages Collect Log File dmesg > system.logF Mellanox Technologies... -
Page 40: Chapter 7 Specifications
The ConnectX-4 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product. b. Typical power for ATIS traffic load. c. For both operational and non-operational states. Mellanox Technologies... -
Page 41: Mcx454A-Fcat Specifications
The ConnectX-4 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product. b. Typical power for ATIS traffic load. c. For both operational and non-operational states. Mellanox Technologies... -
Page 42: Mcx455A-Fcat Specifications
The ConnectX-4 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product. b. Typical power for ATIS traffic load. c. For both operational and non-operational states. Mellanox Technologies... -
Page 43: Mcx456A-Fcat Specifications
The ConnectX-4 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product. b. Typical power for ATIS traffic load. c. For both operational and non-operational states. Mellanox Technologies... -
Page 44: Mcx455A-Ecat Specifications
The ConnectX-4 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product. b. Typical power for ATIS traffic load. c. For both operational and non-operational states. Mellanox Technologies... -
Page 45: Mcx456A-Ecat Specifications
The ConnectX-4 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product. b. Typical power for ATIS traffic load. c. For both operational and non-operational states. Mellanox Technologies... -
Page 46: Airflow Specifications
Solid Yellow Indicates an active physical link Blinking Yellow Indicates a problem with the physical link Solid Green Indicates a valid logical (data activity) link with no active traffic Blinking Green Indicates a valid logical link with active traffic Mellanox Technologies... -
Page 47: Board Mechanical Drawing And Dimensions
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 Board Mechanical Drawing and Dimensions Figure 2: Mechanical Drawing of the Single-port x8 Adapter Cards 142.24 68.90 Figure 3: Mechanical Drawing of the Dual-port x8 Adapter Cards 142.24 68.90 Mellanox Technologies... -
Page 48: Figure 4: Mechanical Drawing Of The Single-Port X16 Adapter Cards
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 Figure 4: Mechanical Drawing of the Single-port x16 Adapter Cards 142.24 68.90 Figure 5: Mechanical Drawing of the Dual-port x16 Adapter Card 142.24 68.90 Mellanox Technologies... -
Page 49: Bracket Mechanical Drawings
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 7.10 Bracket Mechanical Drawings Figure 6: Single-port Bracket Figure 7: Dual-port Bracket Mellanox Technologies... -
Page 50: Appendix A Interface Connectors Pinout
Table 14 - Connector Pin Number and Name to Signal Name Map Connector Pin Number Connector Pin Name Port A Signal Name TXN_2 Tx2n TXP_2 Tx2p TXN_4 Tx4n TXP_4 Tx4p ModSelL_Port0 ModSelL ResetL_Port0 ResetL VccRx RXP_3 Rx3p RXN_3 Rx3n Mellanox Technologies... - Page 51 Connector Pin Number Connector Pin Name Port A Signal Name RXP_1 Rx1p RXN_1 Rx1n RXN_2 Rx2n RXP_2 Rx2p RXN_4 Rx4n RXP_4 Rx4p ModPrsl_Port0 Mod PrsL IntL IntL VccTx Vcc1 LPMode_Port0 LPMode TXP_3 Tx3p TXN_3 Tx3n TXP_1 Tx1p TXN_1 Tx1n Mellanox Technologies...
-
Page 52: Pci Express X8 Orx16 Edge Connector Pinout
PCI Express x8 orx16 Edge Connector Pinout The adapter cards support PCI Express 3.0 x8 or x16 edge connector and the PCI Expressx8 or x16 standard pinout according to the PCI Express 3.0 specification. Figure 9: PCIe x8 Connector Pinout Mellanox Technologies... -
Page 53: Figure 10: Pcie X16 Connector Pinout
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 Figure 10: PCIe x16 Connector Pinout Mellanox Technologies... -
Page 54: Appendix B Finding The Guid/Mac And Serial Number On The Adapter Card
GUID and a MAC (derived from the GUID). The revision indicated on the labels in the following figures do not necessarily repre- sent the latest revision of the card. Figure 11: MCX453A-FCAT Board Label Figure 12: MCX454A-FCAT Board Label Mellanox Technologies... -
Page 55: Figure 13: Mcx455A-Fcat Board Label
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 Figure 13: MCX455A-FCAT Board Label Figure 14: MCX456A-FCAT Board Label Figure 15: MCX455A-ECAT Board Label Mellanox Technologies... -
Page 56: Figure 16: Mcx456A-Ecat Board Label
ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 Figure 16: MCX456A-ECAT Board Label Mellanox Technologies... - Page 57 This equipment should be installed in compliance with local and national electrical codes. 8. Hazardous Radiation Exposure Caution – Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure. Mellanox Technologies...
- Page 58 ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 CLASS 1 LASER PRODUCT and reference to the most recent laser standards: IEC 60 825-1:1993 + A1:1997 + A2:2001 and EN 60825-1:1994+A1:1996+ A2:20. Mellanox Technologies...
- Page 59 L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur. 7. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies...
- Page 60 Mise en garde – l'utilisation de commandes ou de réglages ou l'exécution de procédures autres que ce qui est spécifié dans les présentes peut engendrer une exposition au rayonnement grave. PRODUIT LASER DE CLASSE 1 » et références aux normes laser les plus récentes CEI 60 825-1 Mellanox Technologies...
- Page 61 Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestim- mungen erfolgen. 7. Regionale und nationale elektrische Bestimmungen t Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bes- timmungen installiert werden. This equipment should be installed in compliance with local and national elec- trical codes. Mellanox Technologies...
- Page 62 Rev 1.9 8. Strahlenkontak Achtung – Nutzung von Steuerungen oder Einstellungen oder Ausführung von Prozeduren, die hier nicht spezifiziert sind, kann zu gefährlichem Strahlenkon- takt führen. Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards : ICE 60 825-1 Mellanox Technologies...
- Page 63 Este equipo se debe instalar conforme a los códigos eléctricos locales y nacionales. 8. Exposición a niveles de radiación peligrosos Precaución: el uso de controles o ajustes o la realización de procedimientos distintos de los que aquí se especifican podrían causar exposición a niveles de radiación peligrosos. Mellanox Technologies...
- Page 64 ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1.9 PRODUCTO LÁSER DE CLASE 1 y referencia a las normas de láser más recientes: IEC 60825-1 Mellanox Technologies...
Need help?
Do you have a question about the ConnectX MCX454A-FCAT and is the answer not in the manual?
Questions and answers