Dell QLogic QL45212 User Manual
  1. Manuals
  2. Brands
  3. Dell Manuals
  4. Adapter
  5. QLogic QL45212
  6. User manual
Dell QLogic QL45212 User Manual

Dell QLogic QL45212 User Manual

25gb intelligent ethernet adapter
Hide thumbs
Table of Contents

Advertisement

Quick Links

Download this manual
User's Guide
25Gb Intelligent Ethernet Adapter
QL45212
Third party information brought to
BC0154503-00 B
you courtesy of Dell.

Advertisement

Table of Contents
loading

Related Manuals for Dell QLogic QL45212

Summary of Contents for Dell QLogic QL45212

  • Page 1 User’s Guide 25Gb Intelligent Ethernet Adapter QL45212 Third party information brought to BC0154503-00 B you courtesy of Dell.
  • Page 2 Sections Affected Updated the list of features to replace the list of “Features” on page 1 supported Dell servers with a general reference to the PowerEdge product line. Added NPAR and RoCE to the list of features. Added TSO, LSO, GSO, LOR, RSC, and Microsoft dynamic VMQ and Linux Multiqueue to the list of features.
  • Page 3 “Network Connectivity Testing for Linux” on page 72 Updated instances of automatic port speed negoti- “Dell Z9100 Switch Configuration” on page 75 ation to auto-negotiation. Updated the glossary. “Glossary” on page 78 BC0154503-00 B...

  • Page 4: Table Of Contents

    Table of Contents Preface Overview ........... Intended Audience .
  • Page 5 Installing the Windows Drivers ......Running the Dell Update Package in the GUI ....
  • Page 6 Cisco Nexus 6000 Ethernet Switch ..... . Dell Z9100 Ethernet Switch ......
  • Page 7 User’s Guide—25Gb Intelligent Ethernet Adapter QL45212 Adapter LEDS Dell Z9100 Switch Configuration Glossary Index BC0154503-00 B...
  • Page 8 Dell Update Package Splash Screen ........
  • Page 9 QLogic QL45212 Linux Drivers ........

  • Page 10: Preface

    This user’s guide describes installation, configuration, and management of the ® QLogic QL45212 25Gb Intelligent Ethernet Adapter. Intended Audience This guide is intended for system administrators and other technical staff ® members responsible for configuring and managing adapters installed on Dell ® ® ® ® PowerEdge servers in Windows...

  • Page 11: Documentation Conventions

     Appendix A, Adapter LEDS describes the adapter LEDs and their significance.  Appendix B, Dell Z9100 Switch Configuration describes how to configure the Dell Z9100 switch port for 25Gbps. Documentation Conventions This guide uses the following documentation conventions: NOTE ...

  • Page 12: License Agreements

    Dell provides several online and telephone-based support and service options. Availability varies by country and product, and some services may not be available in your area. To contact Dell for sales, technical support, or customer service issues: Go to support.dell.com.

  • Page 13: Laser Safety

    Preface Legal Notices Laser Safety FDA Notice This product complies with DHHS Rules 21CFR Chapter I, Subchapter J. This product has been designed and manufactured according to IEC60825-1 on the safety label of laser product. CLASS I LASER Class 1 Laser Product Caution—Class 1 laser radiation when open.

  • Page 14: Agency Certification

    Preface Legal Notices Agency Certification The following sections summarize the EMC and EMI test specifications performed on the QL45212 Intelligent Ethernet Adapter to comply with emission, immunity, and product safety standards: EMI and EMC Requirements FCC Part 15 compliance: Class A FCC compliance information statement: This device complies with Part 15 of the FCC Rules.

  • Page 15: Kcc: Class A

    Preface Legal Notices KCC: Class A Korea RRA Class A Certified Product Name/Model: Converged Network Adapters and Intelligent Ethernet Adapters Certification holder: QLogic Corporation Manufactured date: Refer to date code listed on product Manufacturer/Country of origin: QLogic Corporation/USA A class equipment As this equipment has undergone EMC registration for business purpose, the seller and/or the buyer is (Business purpose info/telecom-...

  • Page 16: Product Overview

    Adapter Specifications Functional Description The QL45212 Adapter is a 25Gb Intelligent Ethernet Adapter that is designed to perform accelerated data networking for Dell PowerEdge systems. The QL45212 Adapter includes a 25Gb Ethernet MAC with full-duplex capability. Features The QL45212 Adapter provides the following features: ...
  • Page 17 1–Product Overview Features  ® Microsoft dynamic VMQ and Linux multiqueue  Adaptive interrupts  Receive side scaling (RSS)  Transmit side scaling (TSS)  Stateless offloads for NVGRE/VXLAN tunneled traffic  Manageability  System management bus (SMB) controller  Advanced Configuration and Power Interface (ACPI) 1.1a compliant (multiple power modes) ...

  • Page 18: Adaptive Interrupt Frequency

    1–Product Overview Adapter Specifications Adaptive Interrupt Frequency The adapter driver intelligently adjusts host interrupt frequency based on traffic conditions to increase overall application throughput. When traffic is light, the adapter driver interrupts the host for each received packet, minimizing latency. When traffic is heavy, the adapter issues one host interrupt for multiple, back-to-back incoming packets, preserving host CPU cycles.

  • Page 19: Installing The Hardware

    Installing the Hardware  System Requirements  Safety Precautions  Preinstallation Checklist  Installing the Adapter BC0154503-00 B...

  • Page 20: System Requirements

    2–Installing the Hardware System Requirements System Requirements Before you install a QLogic QL45212 Adapter, verify that your system meets the following hardware (Table 2-1) and operating system (Table 2-2) requirements: Table 2-1. Host Hardware Requirements Hardware Requirement Architecture IA-32 or EMT64 processor that meets operating system...

  • Page 21: Safety Precautions

    Verify that your system is using the latest BIOS. NOTE If you acquired the adapter software on a disk or from the Dell Web Site (support.dell.com), verify the path to the adapter driver files. If your system is active, shut it down.

  • Page 22: Installing The Adapter

    2–Installing the Hardware Installing the Adapter Installing the Adapter The following instructions apply to installing the QL45212 Adapter in most systems. For details about performing these tasks, refer to the documents that were supplied with your system. Review “Safety Precautions” on page 6 “Preinstallation Checklist”...

  • Page 23: Installing Drivers

     Statistics The QL45212 Adapter Linux drivers and supporting documentation are available dell.support.com.Table 3-1 describes the QL45212 Adapter Linux drivers. Table 3-1. QLogic QL45212 Linux Drivers Linux Description Driver The Linux core module manages all PCI device resources (registers, host interface queues, and so on).

  • Page 24: Installing The Linux Drivers Without Roce

    3–Installing Drivers Linux Driver Software Table 3-1. QLogic QL45212 Linux Drivers (Continued) Linux Description Driver qede Linux Ethernet driver for the QL45212 Adapter. This driver directly controls the hardware and is responsible for sending and receiving Ethernet pack- ets on behalf of the Linux host networking stack. This driver also receives and processes device interrupts, on behalf of itself (for L2 networking).

  • Page 25: Removing The Linux Drivers

    3–Installing Drivers Linux Driver Software Removing the Linux Drivers There are two procedures for removing Linux drivers: one for a non-RoCE environment and another for a RoCE environment. Choose the procedure that matches your environment.  Removing Linux Drivers in a non-RoCE Environment ...
  • Page 26 3–Installing Drivers Linux Driver Software Removing Linux Drivers in a RoCE Environment To remove Linux drivers in a RoCE environment: Unload and remove the Linux drivers. modprobe -r qedr modprobe -r qede modprobe -r qed depmod -a Remove driver module files. ...

  • Page 27: Installing Linux Drivers Using The Src Rpm Package

    3–Installing Drivers Linux Driver Software Installing Linux Drivers Using the src RPM Package To install Linux drivers using the src RPM package: Type the following at a command prompt: rpm -ivh qlgc-fastlinq-<version>.src.rpm Change the directory to the RPM path and build the binary RPM for the kernel.

  • Page 28: Installing Linux Drivers Using The Kmp/Kmod Rpm Package

    3–Installing Drivers Linux Driver Software Installing Linux Drivers Using the kmp/kmod RPM Package To install kmod RPM package: Type the following command at a command prompt: rpm -ivh kmod-qlgc-fastlinq-<version>.<arch>.rpm Reload the driver: modprobe -r qede modprobe qede Installing Linux Drivers Using the TAR File To install Linux drivers using the TAR file: Create a directory and extract the TAR files to the directory: tar xjvf fastlinq-minor-<version>.tar.bz2...

  • Page 29: Installing The Linux Drivers With Roce

    Installing Linux Drivers for RoCE in an Out-of-box OFED Environment Installing Linux Drivers for RoCE in an In-box OFED Environment To install Linux drivers in an in-box OFED environment: Download the QL45212 Adapter Linux drivers from dell.support.com. Configure RoCE on the adapter, as described in “Configuring RoCE on the Adapter for Linux”...

  • Page 30: Installing Linux Drivers For Roce In An Out-Of-Box Ofed Environment

    Linux Driver Software Installing Linux Drivers for RoCE in an Out-of-box OFED Environment To install Linux drivers in an out-of-box OFED environment: Download the QL45212 Adapter Linux drivers from dell.support.com. Configure RoCE on the adapter, as described in “Configuring RoCE on the Adapter for Linux”...

  • Page 31: Linux Driver Optional Parameters

    3–Installing Drivers Linux Driver Software Linux Driver Optional Parameters Table 3-2 describes the qede driver optional parameters. Table 3-2. qede Driver Optional Parameters Parameter Description Specifies the debug message level. debug Forces interrupts on all QL45212 adapters in the system using int_mode legacy INTx mode (1) or MSI mode (2).

  • Page 32: Linux Driver Messages

    Managing Adapter Properties  Setting Power Management Options Installing the Windows Drivers You can run a software or driver Dell update package in two ways:  Running the Dell Update Package in the GUI  Running the Dell Update Package from the Command Line...
  • Page 33 3–Installing Drivers Windows Driver Software In the Dell Update Package window (Figure 3-1), click Install. Figure 3-1. Dell Update Package Window ® In the QLogic Super Installer—InstallShield Wizard’s Welcome window (Figure 3-2), click Next. Figure 3-2. QLogic InstallShield Wizard: Welcome Window...
  • Page 34 3–Installing Drivers Windows Driver Software In the wizard’s License Agreement window (Step 3-3): Read the QLogic End User Software License Agreement. To continue, select I accept the terms in the license agreement. Click Next. Figure 3-3. QLogic InstallShield Wizard: License Agreement Window BC0154503-00 B...
  • Page 35 3–Installing Drivers Windows Driver Software Complete the wizard’s Setup Type window (Figure 3-4) as follows: Select one of the following setup types:  Click Complete to install all program features.  Click Custom to manually select the features to be installed. To continue, click Next.
  • Page 36 3–Installing Drivers Windows Driver Software If you selected Custom in Step 5, complete the Custom Setup window (Figure 3-5) as follows: Select the features to install. By default, all features are selected. To change a feature’s install setting, click the icon next to it, and then select one of the following options: ...
  • Page 37 3–Installing Drivers Windows Driver Software In the InstallShield Wizard’s Ready To Install window (Figure 3-6), click Install. The InstallShield Wizard installs the QLogic Adapter drivers and Management Software Installer. Figure 3-6. InstallShield Wizard: Ready to Install the Program Window When the installation is complete, the InstallShield Wizard Completed window appears (Figure 3-7).

  • Page 38: Running The Dell Update Package From The Command Line

    Figure 3-8. Dell Update Package Window Running the Dell Update Package from the Command Line Running the Dell update package from the command line, with no options specified, results in the same behavior as double-clicking the icon representing the Dell update package.

  • Page 39: Installation Options

    To install or update only the driver components: /driveronly NOTE This command requires the /s option.  To define a specific path for the Dell update package’s log file: /l=<path> NOTE This option cannot be used in combination with the advanced installation options, /passthrough or /capabilities.

  • Page 40: Examples

    /passthrough directly to the QLogic installation software of the Dell update package. This mode suppresses any provided GUIs, but not necessarily those of the QLogic software. /passthrough  (Advanced) To return a coded description of this Dell update package's supported features: /capabilities NOTE This command requires the /s option.

  • Page 41: Managing Adapter Properties

    3–Installing Drivers Windows Driver Software Managing Adapter Properties To view or change the QL45212 Adapter properties In Control Panel, click Device Manager. Click the Advanced section of the chosen port. Setting Power Management Options You can set power management options to allow the operating system to turn off the controller to save power or to allow the controller to wake up the computer.

  • Page 42: Vmware Driver Software

    3–Installing Drivers VMware Driver Software VMware Driver Software This section describes the qedentv VMware ESXi driver for the QL45212 Adapter.  Installing VMware Drivers  VMware Driver Optional Parameters  VMware Driver Parameter Defaults  Removing the VMware Driver Installing VMware Drivers You can use the driver zip file to install a new driver or update an existing driver.

  • Page 43: Vmware Driver Optional Parameters

    3–Installing Drivers VMware Driver Software To upgrade an existing driver, follow the steps for a new installation, except replace the command in Step 6 with the following: # esxcli software vib update -v /tmp/qedentv-1.0.3.11-1OEM.550.0.0.1331820.x86_64.vib VMware Driver Optional Parameters Table 3-4 describes the optional parameters that can be supplied as a command line arguments to the esxcfg-module command.
  • Page 44 3–Installing Drivers VMware Driver Software Table 3-4. VMware Driver Optional Parameters (Continued) Parameter Description The RSS parameter specifies the number of receive side scal- ing queues used by the host or virtual extensible LAN (VxLAN) tunneled traffic for a PF. RSS can be 2, 3, or 4, or one of the following values: ...

  • Page 45: Vmware Driver Parameter Defaults

    3–Installing Drivers VMware Driver Software VMware Driver Parameter Defaults Table 3-5 lists the VMware driver parameter defaults. Table 3-5. VMware Driver Parameter Defaults Parameter Default Speed Auto-negotiation with speed advertised Flow Control Auto-negotiation with Rx and Tx advertised 1,500 (range 46–9,600) Rx Ring Size 8,192 (range 128–8,192) Tx Ring Size...

  • Page 46: Upgrading Firmware

    Upgrading Firmware The firmware Dell update package is a Flash update utility only (it is not used for adapter configuration). You run the firmware Dell update package by double-clicking the executable file. Alternatively, the firmware Dell update package can be run from the command line and supports a number of command line options.
  • Page 47 4–Upgrading Firmware Run by Double-Clicking Follow the on-screen instructions. Click Yes to continue the installation, as shown in Figure 4-2. Figure 4-2. Continue Dell Update Package Installation BC0154503-00 B...
  • Page 48 4–Upgrading Firmware Run by Double-Clicking The installer indicates that it is loading the new firmware, as shown in Figure 4-3. Figure 4-3. Loading New Firmware BC0154503-00 B...
  • Page 49 4–Upgrading Firmware Run by Double-Clicking When complete, the installer indicates the result of the installation, as shown Figure 4-4. Click Yes to reboot. Figure 4-4. Result of Installation Click Finish to complete the installation, as shown in Figure 4-5. Figure 4-5. Finish Installation BC0154503-00 B...

  • Page 50: Run From The Command Line

    Running the firmware Dell update package from the command line, with no options specified, results in the same behavior as double-clicking the Dell update package icon. Note that the actual file name of the Dell update package will vary. C:\> Network_Firmware_2T12N_WN32_<version>_X16.EXE...

  • Page 51: Preboot Adapter Configuration

    Preboot Adapter Configuration Getting Started During the host boot process, you have the opportunity to pause and perform adapter management tasks using the Human Infrastructure Interface (HII) application. These tasks include the following:  Displaying Firmware Image Properties  Device Level Configuration ...
  • Page 52 5–Preboot Adapter Configuration Getting Started To start the HII application: Open the System Setup page for your platform. For information about launching the System Setup, consult the user guide for your platform. In the System Setup page, select Device Settings, and then click Finish. In the Device Settings page, select the QL45212 adapter port that you want to configure.
  • Page 53 5–Preboot Adapter Configuration Getting Started The Main Configuration Page presents the adapter management options, as shown in Figure 5-2. Figure 5-1. Main Configuration Page In addition to the management options, the Main Configuration Page presents the adapter properties shown in Table 5-1.

  • Page 54: Displaying Firmware Image Properties

    5–Preboot Adapter Configuration Displaying Firmware Image Properties Displaying Firmware Image Properties To display firmware image properties, select Firmware Image Properties from the Main Configuration Page (Figure 5-2). The Firmware Image Properties page presents the following:  Family Firmware Version: multiboot image version, which comprises several firmware images.
  • Page 55 5–Preboot Adapter Configuration Device Level Configuration To enable NIC partitioning (NPAR), single root input/output virtualization (SR-IOV), or both: Select Device Level Configuration in the Main Configuration Page (Figure 5-1), and then click Finish. In the Device Level Configuration page (Figure 5-2), click the Virtualization Mode drop-down, and choose from the following options: ...

  • Page 56: Nic Configuration

    5–Preboot Adapter Configuration NIC Configuration NIC Configuration To configure the port link speed: Select NIC Configuration in the Main Configuration Page (Figure 5-1), and then click Finish. In the NIC Configuration page, select from the link speed options listed in Table 5-2.

  • Page 57: Data Center Bridging Configuration

    5–Preboot Adapter Configuration Data Center Bridging Configuration Data Center Bridging Configuration The data center bridging settings comprise the DCBX protocol and the remote direct memory access (RDMA) over converged Ethernet (RoCE) priority. To configure the data center bridging (DCB) settings: Select Data Center Bridging (DCB) Settings in the Main Configuration Page (Figure...
  • Page 58 5–Preboot Adapter Configuration NIC Partitioning Configuration In the Global Bandwidth Allocation page (Figure 5-3), click on each partition minimum and maximum TX bandwidth field for which you want to allocate bandwidth. There are four partitions per port if NPAReP mode is disabled; eight partitions per port if NPAReP mode is enabled.

  • Page 59: Configuring Roce

    Configuring RoCE This chapter describes RoCE configuration on the QL45212 Adapter, the Ethernet switch, and the Windows or Linux host. This chapter describes the following topics:  Supported Operating Systems and OFED  Planning  Preparing the Adapter  Preparing the Ethernet Switch ...

  • Page 60: Planning

    6–Configuring RoCE Planning Planning As you prepare to implement RoCE, consider the following limitations:  If you are using the in-box OFED, the operating system should be the same on the server and client systems. Some of the applications may work between different operating systems, but there is no guarantee.

  • Page 61: Preparing The Adapter

    Preparing the Ethernet Switch ® ® This section describes how to configure a Cisco Nexus 6000 Ethernet Switch and a Dell Z9100 Ethernet switch for RoCE.  Cisco Nexus 6000 Ethernet Switch  Dell Z9100 Ethernet Switch Cisco Nexus 6000 Ethernet Switch...
  • Page 62 6–Configuring RoCE Preparing the Ethernet Switch Configure queuing class maps. switch(config)# class-map type queuing class-roce switch(config)# match qos-group 3 Configure network QoS class maps. switch(config)# class-map type network-qos class-ro switch(config)# match qos-group 3 Configure QoS policy maps. switch(config)# policy-map type qos roce switch(config)# class type qos class-roc switch(config)# set qos-group 3 Configure queuing policy maps to assign network bandwidth.

  • Page 63: Dell Z9100 Ethernet Switch

    Preparing the Ethernet Switch Dell Z9100 Ethernet Switch Configuring the Dell Z9100 Ethernet Switch for RoCE comprises configuring a DCB map for RoCE, configuring priority-based flow control (PFC) and enhanced transmission selection (ETS), verifying the DCB map, applying the DCB map to the port, verifying PFC and ETS on the port, specifying the DCB protocol, and assigning a VLAN ID to the switch port.
  • Page 64 BW-COMMITTED BW-PEAK TSA Rate(Mbps) Burst(KB) Rate(Mbps) Burst(KB) ------------------------------------------------------------------------ 0,1,2,3,5,6,7 Dell(Conf)# do show interfaces twentyFiveGigE 1/5/1 pfc detail Interface twentyFiveGigE 1/5/1 Admin mode is on Admin is enabled, Priority list is 4 Remote is enabled, Priority list is 4 Remote Willing Status is enabled...
  • Page 65 66 Input Appln Priority TLV pkts, 99 Output Appln Priority TLV pkts, 0 Error Appln Priority TLV Pkts Configure the DCBX protocol (CEE in this example). Dell(conf)# interface twentyFiveGigE 1/8/1 Dell(conf-if-tf-1/8/1)# protocol lldp Dell(conf-if-tf-1/8/1-lldp)# dcbx version cee Assign a VLAN ID to the switch port to match the VLAN ID assigned to the adapter (5).

  • Page 66: Configuring Roce On The Adapter For Windows Server

    6–Configuring RoCE Configuring RoCE on the Adapter for Windows Server Configuring RoCE on the Adapter for Windows Server Configuring RoCE on the adapter for Windows Server comprises enabling RoCE on the adapter and verifying the Network Direct MTU size. To configure RoCE on a Windows Server host: Enable RoCE on the adapter.
  • Page 67 6–Configuring RoCE Configuring RoCE on the Adapter for Windows Server Select Network Direct Mtu Size from the Property menu. The network direct MTU size must be less than the jumbo packet size. In this example, select 1024. Click OK. ® Verify that RoCE is enabled on the adapter using Windows PowerShell The Get-NetAdapterRdma command lists the adapters that support RDMA—both ports are enabled.
  • Page 68 6–Configuring RoCE Configuring RoCE on the Adapter for Windows Server Connect a server message block (SMB) drive, run RoCE traffic, and verify the results. Map an SMB drive, and then run RoCE traffic. Launch Performance Monitor. Select <Local computer> under Select counters from computer:. Select Add RDMA Activity, and then select <All instances>...
  • Page 69 6–Configuring RoCE Configuring RoCE on the Adapter for Windows Server Verify the SMB connection using Windows PowerShell. Type the net use and netstat -xan commands to show Share1 is mapped as SMB share. PS C:\Users\Administrator> net use Status Local Remote Network ------------------------------------------------------------- \\192.168.10.10\Share1 Microsoft Window...

  • Page 70: Configuring Roce On The Adapter For Linux

    6–Configuring RoCE Configuring RoCE on the Adapter for Linux Configuring RoCE on the Adapter for Linux This section describes the RoCE configuration procedure for RHEL and SLES. It also describes how to verify the RoCE configuration and provides some guidance about using global IDs (GIDs) with VLAN interfaces.

  • Page 71: Installing Out-Of-Box Ofed For Rhel

    6–Configuring RoCE Configuring RoCE on the Adapter for Linux Installing Out-of-box OFED for RHEL To install out-of-box OFED for RHEL: Download latest out-of-box OFED from the following link: http://downloads.openfabrics.org/OFED. Remove the existing Linux drivers, as described in “Removing Linux Drivers in a RoCE Environment”...

  • Page 72: Roce Configuration For Sles

    6–Configuring RoCE Configuring RoCE on the Adapter for Linux RoCE Configuration for SLES To configure RoCE on the adapter for an SLES host, OFED must be installed and configured on the SLES host. Some SLES hosts come with OFED installed (in-box), and some do not (out-of-box).

  • Page 73: Verifying The Roce Configuration On Linux

    6–Configuring RoCE Configuring RoCE on the Adapter for Linux Compile and install the FastLinQ drivers from the TAR file as described in “Installing Linux Drivers for RoCE in an Out-of-box OFED Environment” on page When installing OFED 3.18, if the qedr compilation fails because of a compact.h file error, type the following commands: >...
  • Page 74 6–Configuring RoCE Configuring RoCE on the Adapter for Linux Configure the IP address and enable the port using a configuration method, such as ifconfig: # ifconfig ethX 192.168.10.10/24 up Type the ibv_devinfo command. For each PCI function, you should see a separate hca_id, as shown in the following example: root@captain:~# ibv_devinfo hca_id: qedr0...
  • Page 75 6–Configuring RoCE Configuring RoCE on the Adapter for Linux The following are examples of successful ping pong tests on the server and the client: Server: root@captain:~# ibv_rc_pingpong -d qedr0 -g 0 local address: LID 0x0000, QPN 0xff0000, PSN 0xb3e07e, GID fe80::20e:1eff:fe50:c7c0 remote address: LID 0x0000, QPN 0xff0000, PSN 0x934d28, GID fe80::20e:1eff:fe50:c570...

  • Page 76: Vlan Interfaces And Gid Index Values

    6–Configuring RoCE Configuring RoCE on the Adapter for Linux VLAN Interfaces and GID Index Values If you are using VLAN interfaces on both server and client, then configure the same VLAN ID on the switch. If you are running traffic through a switch, the Infiniband applications must use the correct GID value, which is based on the VLAN ID and VLAN IP address.

  • Page 77: Iscsi Extensions For Rdma

    iSCSI Extensions for RDMA This chapter describes the procedure for configuring iSCSI Extensions for RDMA (iSER) for RHEL and SLES. As you prepare to configure iSER, consider the following:  iSER is supported only in INBOX OFED for following operating systems: ...
  • Page 78 7–iSCSI Extensions for RDMA Configuring iSER for RHEL Load the RDMA services. systemctl start rdma modprobe qed modprobe qede modprobe qedr modprobe ib_iser modprobe ib_isert Verify all RDMA and iSER modules loaded on the initiator and target devices using the lsmod | grep qed and lsmod | grep iser commands. Verify that there are separate hca_id instances by typing the ibv_devinfo command, as described in Step 6...
  • Page 79 7–iSCSI Extensions for RDMA Configuring iSER for RHEL You can use a Linux TCM-LIO target with which to test iSER. The setup is the same for any iSCSI target, except you type command enable_iser Boolean=true on the applicable portals. The portal instances are identified as "iser"...
  • Page 80 7–iSCSI Extensions for RDMA Configuring iSER for RHEL Confirm that the Iface Transport is iser in the target connection, as shown in the following figure. Type a command such as the following: iscsiadm -m session -P2 Check for a new iSCSI device, as shown in the following figure. Type command such as the following: lsscsi BC0154503-00 B...

  • Page 81: Configuring Iser For Sles12 Linux

    7–iSCSI Extensions for RDMA Configuring iSER for SLES12 Linux Configuring iSER for SLES12 Linux The TargetCLI is not in-box on SLES12, therefore you must do the following: To install TargetCLI, copy and install the following RPMs from ISO image (x86_64 and noarch location). lio-utils-4.1-14.6.x86_64.rpm python-configobj-4.7.2-18.10.noarch.rpm python-PrettyTable-0.7.2-8.5.noarch.rpm...

  • Page 82: Set Cpus To Maximum Performance Mode

    7–iSCSI Extensions for RDMA Optimizing Linux Performance Set CPUs to Maximum Performance Mode Set the CPU scaling governor to performance by using the following script to set all CPUs to maximum Performance mode for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue;...

  • Page 83: Block Device Staging

    7–iSCSI Extensions for RDMA Optimizing Linux Performance Block Device Staging Set the block device staging settings for each iSCSI device/target: echo noop > /sys/block/sdd/queue/scheduler echo 2 > /sys/block/sdd/queue/nomerges echo 0 > /sys/block/sdd/queue/add_random echo 1 > /sys/block/sdd/queue/rq_affinity BC0154503-00 B...

  • Page 84: Troubleshooting

    Troubleshooting This chapter provides information about the following topics:  Troubleshooting Checklist  Verifying that Current Drivers are Loaded  Testing Network Connectivity  Microsoft Virtualization with Hyper-V  Linux  Miscellaneous Troubleshooting Checklist CAUTION Before you open the server cabinet to add or remove the adapter, review “Safety Precautions”...

  • Page 85: Verifying That Current Drivers Are Loaded

    8–Troubleshooting Verifying that Current Drivers are Loaded  Replace the failed adapter with one that is known to work properly. If the second adapter works in the slot where the first one failed, the original adapter is probably defective.  Install the adapter in another functioning system, and then run the tests again.

  • Page 86: Testing Network Connectivity

    8–Troubleshooting Testing Network Connectivity If you loaded a new driver, but have not yet booted, the modinfo command will not show the updated driver information. Instead, type the following dmesg command to view the logs. In this example, the last entry identifies the driver that will be active upon reboot.

  • Page 87: Network Connectivity Testing For Linux

    8–Troubleshooting Microsoft Virtualization with Hyper-V  ® Windows Device Manager: Network Adapter, Advanced Properties—Speed & Duplex settings  ® Windows PowerShell Set-NetAdapterAdvancedProperty –Name “<port name>” –DisplayName “Speed & Duplex” –DisplayValue “10 Gbps Full Duplex Network Connectivity Testing for Linux To verify that the Ethernet interface is up and running, run ifconfig to check the status of the Ethernet interface.

  • Page 88: Miscellaneous

    8–Troubleshooting Miscellaneous Miscellaneous Problem: The QL45212 Adapter has shut down, and an error message appears indicating that the fan on the adapter has failed. Solution: The QL45212 Adapter shut down to prevent permanent damage. Contact QLogic Technical Support for assistance. BC0154503-00 B...
  • Page 89 Adapter LEDS Network link and activity status are indicated by the Link and Activity LEDs located adjacent to the port connector, as described in Table A-1. Table A-1. Port LED Indications Port LED LED Appearance Network State Link LED No link (cable discon- nected) Continuously illuminated Link...
  • Page 90 However, until the auto-negotiation process is standardized, the switch must be explicitly configured to connect to the adapter at 25Gbps. To configure a Dell Z9100 switch port to connect to the QL45212 Adapter at 25Gbps: Establish a serial port connection between your management workstation and the switch.
  • Page 91 Dell(conf)#stack-unit 1 port 5 portmode quad ? speed Each port speed in quad mode Dell(conf)#stack-unit 1 port 5 portmode quad speed ? Quad port mode with 10G speed Quad port mode with 25G speed Dell(conf)#stack-unit 1 port 5 portmode quad speed 25G For information about changing the adapter link speed, see “Testing Network...
  • Page 92 1/5/1 no ip address mtu 9416 switchport flowcontrol rx on tx on no shutdown no intf-type cr4 autoneg For more information about configuring the DZ9100 switch, refer to the Dell Z9100 Switch Configuration Guide at support.dell.com. BC0154503-00 B...
  • Page 93 Glossary adapter data center bridging (DCB) The board that interfaces between the Provides enhancements to existing 802.1 host system and the target devices. bridge specifications to satisfy the require- Adapter is synonymous with Host Bus ments of protocols and applications in the Adapter, host adapter, and board.
  • Page 94 User’s Guide—25Gb Intelligent Ethernet Adapter QL45212 enhanced transmission selection (ETS) Institute of Electrical and Electronics Engineers (IEEE) A standard that specifies the enhancement of transmission selection to support the An international nonprofit organization for allocation of bandwidth among traffic the advancement of technology related to electricity.
  • Page 95 User’s Guide—25Gb Intelligent Ethernet Adapter QL45212 quality of service (QoS) unified extensible firmware interface (UEFI) Refers to the methods used to prevent A specification detailing an interface that bottlenecks and ensure business conti- helps hand off control of the system for the nuity when transmitting data over virtual preboot environment (that is, after the ports by setting priorities and allocating...
  • Page 96 RoCE DCB map 48, installation DCBX LEDs enable management protocol properties (Windows) debug parameter 16, adaptive interrupt frequency Dell update package address, MAC command line address, PCI architecture device auto_fw_reset parameter auto-negotiation name device level configuration Device Manage direct attach copper, See DAC.
  • Page 97 User’s Guide—25Gb Intelligent Ethernet Adapter QL45212 driver installation kmod RPM hardware installation Linux with RoCE HII 36, Linux without RoCE human interface infrastructure, See HII. RoCE, in-box OFED hw_vlan parameter 28, RoCE, out-of-box OFED Hyper-V source RPM TAR file VMware Windows command line Windows GUI IEEE protocol...
  • Page 98 User’s Guide—25Gb Intelligent Ethernet Adapter QL45212 parameters, Linux partitions per port network QoS class PCIe QoS class Performance Monitor QoS policy performance, RoCE queuing class max_vfs parameter maximum transmission unit, See MTU. physical characteristics memory physical function, See PF. messages policy map 46, port speed MSI-X...
  • Page 99 User’s Guide—25Gb Intelligent Ethernet Adapter QL45212 RoCE transmit send offload, See TSO. configuration for RHEL troubleshooting configuration for SLES TSO 16, 29, driver Tx Ring Size 16, limitations Linux driver removal priority supported operating systems, OFED UEFI verification Unified Extensible Firmware Interface, See RPM package 12, UEFI.
  • Page 101 International Offices UK | Ireland | Germany | France | India | Japan | China | Hong Kong | Singapore | Taiwan | Israel © 2015, 2016 QLogic Corporation. All rights reserved worldwide. QLogic and the QLogic logo are registered trademarks of QLogic Corporation. Dell, the Dell logo, and PowerEdge are registered trademarks of Dell, Inc.

Table of Contents