Mellanox Technologies Innova IPsec User Manual
  1. Manuals
  2. Brands
  3. Mellanox Technologies Manuals
  4. Adapter
  5. Innova IPsec
  6. User manual
Mellanox Technologies Innova IPsec User Manual

Mellanox Technologies Innova IPsec User Manual

Ethernet adapter card
Hide thumbs
Table of Contents

Advertisement

Quick Links

Mellanox Innova™ IPsec
Ethernet Adapter Card
User Manual
Rev 1.8
www.mellanox.com
Mellanox Technologies

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the Innova IPsec and is the answer not in the manual?

Questions and answers

Related Manuals for Mellanox Technologies Innova IPsec

Summary of Contents for Mellanox Technologies Innova IPsec

  • Page 1 Mellanox Innova™ IPsec Ethernet Adapter Card User Manual Rev 1.8 www.mellanox.com Mellanox Technologies...
  • Page 2 KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES...

  • Page 3: Table Of Contents

    3.7 Identify the Card in Your System....... . . 20 Chapter 4 Mellanox Innova IPsec Offload Overview ....21 4.1 Security Engines and IPsec Protocols .
  • Page 4 4.2.1 Mellanox Innova IPsec Ethernet Driver Module..... 22 4.2.2 mlx5_fpga_tools Module ........23 4.2.3 Key Generation and Exchange .
  • Page 5 9.3 Mellanox Innova IPsec EN LEDs ........

  • Page 6: List Of Tables

    MNV101511A-BCIT Mellanox Innova IPsec Passive Cooling Adapter Card..11 Table 3: MNV101512A-BCIT Mellanox Innova IPsec Active Cooling Adapter Card..11 Table 4: Features ............12 Table 5: Documents List.

  • Page 7: List Of Figures

    List of Figures Figure 1: Mellanox Innova IPsec EN Adapter Card Block Diagram ......14 Figure 2: IPsec Solution Layers and Components .

  • Page 8: Revision History

    Section 1.4, “Operating Systems/Distributions,” on page 14. • Updated Figure 3, “MNV101511A-BCIT/MNV101512A-BCIT LEDs Placement (Example),” on page 50. • Mellanox Innova is a trademark of Mellanox Technologies Ltd. February • Updated Section 5.2.2, “Installing the Kernel and Driver,” on page 34 2018 •...
  • Page 9 Burning Tool,” on page 42 • Updated Section 6.3.1.2, “Loading Tool,” on page 42 • Updated Chapter 7,“Updating Mellanox Innova IPsec Adapter Card Firm- ware” on page 44 • Added Figure 7, “Single-Port Short Bracket,” on page 55 •...

  • Page 10: Chapter 1 Introduction

    IPsec algorithms consumes expensive CPU cycles and limits network connection performance. The Mellanox Innova IPsec EN adapter offloads the processing of the IPsec algorithms, frees u p the CPU, and eases network bottlenecks. The adapter integrates advanced network capabilities and encryption offloading in one card, utilizing only a single PCIe slot for both networking and crypto functions.

  • Page 11: Product Overview

    This section provides the ordering part number, port speed, number of ports, and PCI Express speed for the various models. Table 2 - MNV101511A-BCIT Mellanox Innova IPsec Passive Cooling Adapter Card MNV101511A-BCIT - HHHL card with Xilinx Kintex® Ordering Part Number (OPN) UltraScale™...

  • Page 12: Features And Benefits

    (GENEVE, MPLS, QinQ, and so on). With the Mellanox Innova IPsec adapter, data center operators can achieve native performance in the new network architecture.
  • Page 13 Mellanox Innova IPsec adapter supports RoCE specifications delivering low- latency and high- performance over Ethernet networks. Leveraging data center RDMA and RDMA bridging (DCB) capabilities as well as Mellanox Innova IPsec adapter advanced over Converged Ethernet (RoCE) congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.

  • Page 14: Block Diagram

    Block Diagram Figure 1: Mellanox Innova IPsec EN Adapter Card Block Diagram x 8 P C I e G e n 3 x 8 P C Ie G e n 3 J T A G / G P I O...

  • Page 15: Connectivity

    Mellanox drivers. Document no. 3368 http://www.mellanox.com/related-docs/prod_software/ Performance_Tuning_Guide_for_Mellanox_Network_Adapters .pdf MellanoxO FED for Linux Driver Release notes for Mellanox Technologies' MLNX_OFED for Release Notes Linux driver kit for Mellanox adapter cards: http://www.mellanox.com => Products => Software => InfiniBand/VPI Drivers => Mellanox OpenFabrics Enterprise Distribution for Linux (MLNX_OFED) =>...

  • Page 16: Chapter 2 Interfaces

    The adapter cards include special circuits to protect from ESD shocks to the card/server when plugging copper cables. Ethernet QSFP Interface The network port of the Mellanox Innova IPsec adapter is compliant with the IEEE 802.3 Ethernet standards listed in Table 4, “Features,” on page 12.

  • Page 17: Chapter 3 Hardware Installation

    Section 3.5, “Card Installa- tion Instructions,” on page If you need to replace it with the short bracket that is included in the shipping box, please follow the instructions in this section. Rev 1.8 Mellanox Technologies...

  • Page 18: Removing The Existing Bracket

    3. Applying even pressure at both corners of the card, insert the adapter card into the slot until it is firmly seated. When the adapter is properly seated, the adapter port connectors are aligned with the slot opening, and the adapter faceplate is visible against the system chassis. Mellanox Technologies Rev 1.8...

  • Page 19: Cables And Modules

    See Section 9.3, “Mella- nox Innova IPsec EN LEDs,” on page 4. After plugging in a cable, lock the connector using the latching mechanism particular to the cable vendor. When a logical connection is made, the Green LED will light. When data is being transferred the Green LED will blink.

  • Page 20: Identify The Card In Your System

    Identify the Card in Your System Get the device location on the PCI bus by running lspci and locating lines with the string “Mella- nox Technologies”: lspci |grep -i Mellanox Network controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Mellanox Technologies Rev 1.8...

  • Page 21: Chapter 4 Mellanox Innova Ipsec Offload Overview

    Mellanox Innova IPsec Offload Overview Mellanox Innova IPsec Offload Overview The Mellanox Innova IPsec EN adapter is pre-programmed with a Mellanox IPsec offload FPGA logic, offering encryption, decryption and authentication for IPsec security protocol suite. The IPsec offload solution offers three major benefits: 1.

  • Page 22: Offloaded Ipsec Protocols And Internet Protocols

    Linux kernel. 4.2.1 Mellanox Innova IPsec Ethernet Driver Module The Mellanox Innova IPsec adapter has a dedicated driver in the form of a kernel module, mlx5_core.ko. The driver performs the following: • Configures the offload settings and modes in HW.

  • Page 23: Mlx5_Fpga_Tools Module

    The module depends on the mlx5_core module. 4.2.3 Key Generation and Exchange The Mellanox Innova IPsec adapter currently supports offloading of the encryption, decryption and authentication of IPsec traffic. The key generation and exchange protocol, whether done manually or through IKE protocol, remains within complete ownership of the userspace software that is used for IPsec connection creation and management (such as iproute2, libreswan, strongswan and others) and is not affected by the HW or the supplied IPsec kernel module.

  • Page 24: Ipsec Offload For Dpdk Applications

    (data path). The data path is still done with kernel network stack bypass, providing the application with the benefits of both DPDK acceleration and security offload (encryption/decryption). Please refer to Mellanox Innova IPsec EN Release Notes for supported versions. Mellanox Technologies Rev 1.8...

  • Page 25: Chapter 5 Ipsec Offload Software Installation And Operation

    OFED installation on top of the kernel module that is provided in this bundle is not supported. As of version 4.2, MLNX_OFED supports Mellanox Innova IPsec EN adapter card. This type of installation is applicable to RedHat 7.1, 7.2, 7.3 and 7.4 operating systems and Kernel 4.13.
  • Page 26 [-n|--name] Name of the package to be created. [-y|--yes] Answer "yes" to all questions [--force] Force removing packages that depends on MLNX_OFED 1. The firmware will not be updated if you run the install script with the ‘--without-fw-update’ option. Mellanox Technologies Rev 1.8...
  • Page 27 Install packages required by VMA to support VPI --vma|--vma-vpi Install packages required by VMA to work over Ethernet --vma-eth Set configuration for VMA use (to be used with any --with-vma installation parameter). Install packages required by guest os --guest Rev 1.8 Mellanox Technologies...

  • Page 28: Installation Procedure

    Note that all other Mellanox, OEM, OFED, or Distribution IB packages will be removed. Uninstalling the previous version of MLNX_OFED_LINUX Starting MLNX_OFED_LINUX-x.x.x installation ....Installation finished successfully. Attempting to perform Firmware update... Querying Mellanox devices firmware ... Mellanox Technologies Rev 1.8...
  • Page 29 Most of the Mellanox OFED components can be configured or reconfigured after the installation by modifying the relevant configuration files. See the relevant chapters in this manual for details. The list of the modules that will be loaded automatically upon boot can be found in the /etc/ file. infiniband/openib.conf Rev 1.8 Mellanox Technologies...

  • Page 30: Installation Results

    Upon system boot, the Mellanox drivers will be loaded automatically.  To prevent automatic load of the Mellanox drivers upon system boot: Add the following lines to the file. Step 1. "/etc/modprobe.d/mlnx.conf" blacklist mlx4_core blacklist mlx4_en blacklist mlx5_core Mellanox Technologies Rev 1.8...

  • Page 31: Mlnxofedinstall Return Codes

    The latest FW and FPGA update package can be downloaded from www.mellanox.com => Products => Ethernet => SmartNICs => Innova IPsec SmartNIC Ethernet => Download tab. You can run the following update script using one of the modes below: /opt/mellanox/mlnx-fw-updater/mlnx_fpga_updater.sh...

  • Page 32: Uefi Secure Boot

    For more information on the script usage, you can run mlnx_fpga_updater.sh -h. Its is recommended to perform firmware and FPGA upgrade on Mellanox Innova IPsec cards using this script only. 5.1.5 UEFI Secure Boot All kernel modules included in MLNX_OFED for RHEL7 are signed with x.509 key to support loading the modules when Secure Boot is enabled.

  • Page 33: Removing Signature From Kernel Modules

    After the signature has been removed, a massage as the below will no longer be pre- sented upon module loading: "Request for unknown module key 'Mellanox Technologies signing key: 61feb074fc7292f958419386ffdd9d5ca999e403' err -11" However, please note that a similar message as the following will still be presented: "my_module: module verification failed: signature and/or required key missing -...

  • Page 34: Installation Of Kernel Module With Ipsec Offload

    Please make sure that the latest FW, FPGA image and MFT versions are installed. Please refer to the Mellanox Innova IPsec EN Adapter Card Release Notes for the latest versions. Once you have obtained the kernel RPM file, the file can be installed by performing the following steps: 1.

  • Page 35: Installing The Customized Iproute2 Utility

    IPsec tunnel offload state. These flags provide the option to enable offload for IPsec SAs. 1. Obtain the customized iproute2 RPM file by contacting Mellanox support (File Name: iproute2-<version>.x86_64.rpm) Rev 1.8 Mellanox Technologies...
  • Page 36 IPsec policies and SAs: Strongswan (which has IPsec offload support as of version 5.5.3), Libreswan (which has IPsec offload support as of version 3.21) and more. Please refer to the release notes of the above mentioned user space applications for more information on IPsec offload support. Mellanox Technologies Rev 1.8...

  • Page 37: Operating The Ipsec Offload

    Operating the IPsec Offload 5.3.1 Loading/Unloading the Module 5.3.1.1 Automatic Load The Mellanox Innova IPsec Ethernet driver, mlx5_core, is loaded automatically by the kernel when a Mellanox Innova IPsec card is installed. 5.3.1.2 Manual Load/Unload 1. Load/unload mlx5_core using one of the following commands:...
  • Page 38 7. Indicates that we are about to define the template of the outer IP header of our tunnel. 8. The tunnel source and destination IP addresses - can be different than the inner packet IP address. Mellanox Technologies Rev 1.8...

  • Page 39: Destroying Ipsec Tunnels

    Number of packets dropped by decryption engine. This can be as a result of having inband metadata in packet or corrupted decryption. ipsec_dec_auth_fail_packets Number of packets dropped by decryption engine due to authentication issue. Rev 1.8 Mellanox Technologies...
  • Page 40 Total amount of failed SA remove commands by FPGA. This can be a result of remove command on invalid SA. ipsec_cmd_drop Total amount of failed commands. This can be a result of failure to parse command. Mellanox Technologies Rev 1.8...

  • Page 41: Chapter 6 Mlx_Fpga Tool

    Tool mlx_fpga Tool mlx_fpga tool allows the user to burn and update a new FPGA image on Mellanox Innova IPsec adapter card. The tool also enables the user to read/write individual registers in the FPGA configuration space. Tool Requirements •...

  • Page 42: Examples Of Mlx_Fpga Usage

    • Load an FPGA image from user configurable flash: # mlx_fpga -d <device> l/load <optional: load options> where <optional: load options> Load FPGA image from factory flash --factory Load FPGA image from user flash [default option] --user Mellanox Technologies Rev 1.8...

  • Page 43: Debugging Tool

    0x900008 31:00:00 Image time of creation. The hex number is actually the decimal value, i.e. 0x00015324 means 01:53:24 in HH:MM:SS: bits [23:16] = hour (00..23) bits [15:8] = minutes (00..59) bits [7:0] = seconds (00..59) Rev 1.8 Mellanox Technologies...

  • Page 44: Chapter 7 Updating Mellanox Innova Ipsec Adapter Card Firmware

    Updating Mellanox Innova IPsec Adapter Card Firmware This section applies only when updating the ConnectX-4 Lx firmware. In order to burn and update the FPGA image, please refer to Chapter 6,“mlx_fpga Tool” on page 41 Each card is shipped with the latest version of qualified ConnectX-4 Lx firmware at the time of manufacturing.
  • Page 45 Updating Mellanox Innova IPsec Adapter Card Firmware b. To burn the firmware, run: mlxburn -d /dev/mst/mt4117_pciconf0 -i <fw.bin> c. To load the firmware, run: mlxfwreset -d /dev/mst/mt4117_pciconf0 reset -y Rev 1.8 Mellanox Technologies...

  • Page 46: Chapter 8 Troubleshooting

    • Check that both the adapter and its link are set to the same speed and duplex set- established tings • Verify the Mellanox Innova IPsec kernel is loaded • Load mlx5_fpga_tools module • Start mlx_fpga tool FPGA not found on mst...

  • Page 47: Linux

    Mellanox Firmware Tool Once installed, run: (MFT) mst start mst status flint –d <mst_device> q ibstat Ports Information lbv_devinfo To download the latest firmware version refer to Firmware Version http://www.mellanox.com/supportdownloader Upgrade /var/log/messages Collect Log File dmesg > system.logF Rev 1.8 Mellanox Technologies...

  • Page 48: Chapter 9 Specifications

    Air flow is measured ~1” from the heat sink between the heat sink and the cooling air inlet. d. Airflow requirements may vary according to ambient temperature and other parameters. Please contact Mellanox technical support if further assistance is needed. Mellanox Technologies Rev 1.8...

  • Page 49: Mnv101511A-Bcit Specifications

    Air flow is measured ~1” from the heat sink between the heat sink and the cooling air inlet. d. Airflow requirements may vary according to ambient temperature and other parameters. Please contact Mellanox technical support if further assistance is needed. Rev 1.8 Mellanox Technologies...

  • Page 50: Mellanox Innova Ipsec En Leds

    Mellanox Innova IPsec EN LEDs Figure 3: MNV101511A-BCIT/MNV101512A-BCIT LEDs Placement (Example) Group A LEDs: Network LEDs - these LEDs indicate the network link status. See Section 9.3.1, “Network LEDs Operation,” on page 51 for details. Group B LEDs: Debug LEDs - indicate memory calibration done, memory BIST done, ConnectX-4 Lx link up is with traffic, Heartbeat and power good.

  • Page 51: Network Leds Operation

    ConnectX Port Traffic - the LED will blink when there is FPGA-ConnectX traffic (TX/SX). Network Port Ready - the LED will be ON when FPGA-Network link is up. Network Port Traffic - the LED will blink when there is FPGA-Network traffic (TX/SX). Rev 1.8 Mellanox Technologies...

  • Page 52: Fpga Load-Flow Debug Leds

    Red - factory default D12 - Configuration Image Selection Green - user image Board Mechanical Drawing and Dimensions All dimensions are in millimeters. All the mechanical tolerances are +/- 0.1mm. Figure 4: Mechanical Drawing of MNV101511A-BCIT 167.65 68.9 Mellanox Technologies Rev 1.8...

  • Page 53: Figure 5: Mechanical Drawing Of Mnv101512A-Bcit

    Specifications Figure 5: Mechanical Drawing of MNV101512A-BCIT 167.65 68.90 Rev 1.8 Mellanox Technologies...

  • Page 54: Bracket Mechanical Drawing

    Bracket Mechanical Drawing Figure 6: Single-Port Tall Bracket 21.6 120.02 Mellanox Technologies Rev 1.8...

  • Page 55: Figure 7: Single-Port Short Bracket

    Specifications Figure 7: Single-Port Short Bracket 22.83 80.3 Rev 1.8 Mellanox Technologies...

  • Page 56: Appendix A Fast Installation And Update

    IPsec => FW & SW. Each card is shipped with the latest version of the qualified FPGA image and firmware at the time of manufacturing. Please download the Mellanox Innova IPsec bundle that matches the FPGA image burned on your card.  To install the kernel: Locate the RPM files in the Kernel folder: Step 1.
  • Page 57 ./Scripts/mlnx_fpga_updater.sh -d /dev/mst/mt4117_pciconf0 ------------------- /dev/mst/mt4117_pciconf0 /dev/mst/mt4117_pciconf1 From this point on the script will install the FPGA image, the FW and will also ask if to Step 5. install the MFT and do a reset at the end. Rev 1.8 Mellanox Technologies...

  • Page 58: Software, Firmware And Tools Update

    To download the bundle, please refer to www.mellanox.com => Products => Smart- Step 1. NICs => Innova IPsec => FW & SW  To install the most updated kernel: Locate the RPM files in the Kernel folder: Step 1. •...

  • Page 59: Ofed Installation With Script

    Example: ./mlnx_fpga_updater.sh -p <Innova_IPsec_extracted_bundle_directory> For more information on the script usage, you can run mlnx_fpga_updater.sh -h. It is recommended to perform a firmware and FPGA upgrade on Mellanox Innova IPsec cards using this script only. Rev 1.8 Mellanox Technologies...

  • Page 60: Appendix B Finding The Mac And Serial Number On The Adapter Card

    MAC for the Ethernet protocol. The revision indicated on the labels in the following figures do not necessarily represent the latest revision of the card. Figure 8: MNV101511A-BCIT Board Label Figure 9: MNV101512A-BCIT Board Label Rev 1.8 Mellanox Technologies...

  • Page 61: Appendix C Safety Warnings

    6. Equipment Disposal Disposal of this equipment should be in accordance to all national laws and regula- tions. 7. Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical codes. Mellanox Technologies Rev 1.8...
  • Page 62 Caution – Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure. CLASS 1 LASER PRODUCT and reference to the most recent laser standards: IEC 60 825-1:1993 + A1:1997 + A2:2001 and EN 60825-1:1994+A1:1996+ A2:20. Rev 1.8 Mellanox Technologies...

  • Page 63: Appendix D Avertissements De Sécurité D'installation (Warnings In French)

    L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur. 7. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies Rev 1.8...
  • Page 64 Mise en garde – l'utilisation de commandes ou de réglages ou l'exécution de procédures autres que ce qui est spécifié dans les présentes peut engendrer une exposition au rayonnement grave. PRODUIT LASER DE CLASSE 1 » et références aux normes laser les plus récentes CEI 60 825-1 Rev 1.8 Mellanox Technologies...

  • Page 65: Appendix E Sicherheitshinweise (Warnings In German)

    6. Geräteentsorgung Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestim- mungen erfolgen. 7. Regionale und nationale elektrische Bestimmungen t Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bes- timmungen installiert werden. Mellanox Technologies Rev 1.8...
  • Page 66 8. Strahlenkontak Achtung – Nutzung von Steuerungen oder Einstellungen oder Ausführung von Prozeduren, die hier nicht spezifiziert sind, kann zu gefährlichem Strahlenkon- takt führen. Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards : ICE 60 825-1 Rev 1.8 Mellanox Technologies...

  • Page 67: Appendix F Advertencias De Seguridad Para La Instalación (Warnings In Spanish)

    La eliminación definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales. 7. Códigos eléctricos locales y nacionales Este equipo se debe instalar conforme a los códigos eléctricos locales y nacionales. Mellanox Technologies Rev 1.8...
  • Page 68 Precaución: el uso de controles o ajustes o la realización de procedimientos distintos de los que aquí se especifican podrían causar exposición a niveles de radiación peligrosos. PRODUCTO LÁSER DE CLASE 1 y referencia a las normas de láser más recientes: IEC 60825-1 Rev 1.8 Mellanox Technologies...

This manual is also suitable for:

Mnv101511a-bcit-hhhl

Table of Contents