1. Manuals
  2. Brands
  3. Chelsio Communications Manuals
  4. PCI Card
  5. DPDK
  6. Installation and user manual

Chelsio Communications DPDK Installation And User Manual

Driver for linux
Hide thumbs
Table of Contents

Advertisement

Quick Links

Download this manual

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the DPDK and is the answer not in the manual?

Questions and answers

Related Manuals for Chelsio Communications DPDK

Summary of Contents for Chelsio Communications DPDK

  • Page 2 For all support related questions please send email to support@chelsio.com Copyright © 2017. Chelsio Communications. All Rights Reserved. Chelsio ® is a registered trademark of Chelsio Communications. All other marks and names mentioned herein may be trademarks of their respective companies. Chelsio DPDK Driver for Linux...
  • Page 3 Document History Version Revision Date 1.0.0 07/08/2015 1.0.1 10/26/2015 1.0.2 04/01/2016 1.0.3 07/12/2016 1.0.4 12/30/2016 1.0.5 01/30/2017 1.0.6 02/24/2017 Chelsio DPDK Driver for Linux...

  • Page 4: Table Of Contents

    Classification and Filtering 7.1. Pre-requisites 7.2. Usage 7.3. Layer 2 example 7.4. Forwarding multi-cast traffic 7.5. Filtering using outer VLAN Software/Driver Unloading Software/Driver Uninstallation 10. Software/Driver Update 11. Appendix 11.1. Chelsio End-User License Agreement (EULA) Chelsio DPDK Driver for Linux...

  • Page 5: Introduction

    Designed for deployment in virtualized data centers, cloud service installations and high performance computing environments, Chelsio adapters bring a new level of performance metrics and functional capabilities to the computer networking industry. This document describes the installation, use and maintenance of the DPDK driver and its various components. 1.1.

  • Page 6: Hardware Requirements

    T520-CR T520-LL-CR T520-SO-CR T520-BT 1.3. Software Requirements The Chelsio DPDK driver has been developed to run on 64-bit Linux platforms. Following is the list of supported distributions: Linux Distribution Driver/Software RHEL 7.2, 3.10.0-327.el7 UIO, VFIO, Filtering RHEL 7.2, 3.10.0-327.el7.ppc64le (POWER8 LE) VFIO* RHEL 7.1, 3.10.0-229.el7...

  • Page 7: Package Contents

    Package Contents The Chelsio DPDK driver package consists of the following files/directories: Makefile: Makefile for building and installing Chelsio DPDK driver and tools. EULA: Chelsio’s End User License Agreement. src: Source code for Chelsio DPDK driver, Kernel driver and Tools.

  • Page 8: Hardware Installation

    Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 198k 198k 491k 0 --:--:-- --:--:-- --:--:-- 626k Done. xiii. Verify if the adapter was installed successfully by using the lspci command: Chelsio DPDK Driver for Linux...
  • Page 9 However, for T5 40G adapters, the association of physical Ethernet ports and their corresponding network device names is opposite. For this adapter, the port nearest to the motherboard will appear as the first network interface. Chelsio DPDK Driver for Linux...

  • Page 10: Software/Driver Installation

    Compile and install the driver using one of the following options:  To install DPDK source, testpmd and Pktgen tool only: [root@host~]# make dpdk_install  To install DPDK source and all the tools provided in the DPDK repository: [root@host~]# make install The above commands will Note ...

  • Page 11: Firmware Update

    Flashing Firmware Configuration File If not done already, load the kernel mode NIC driver (cxgb4): [root@host~]# modprobe cxgb4 ii. Flash the firmware configuration file (for T6 adapters): [root@host~]# cxgbtool <iface> loadcfg /lib/firmware/cxgb4/t6-config.txt For T5 adapters, use t5-config.txt. Note Chelsio DPDK Driver for Linux...

  • Page 12: Software/Driver Loading

    The driver must be loaded by the root user. Any attempt to load as a regular user will fail. UIO Support Follow the steps mentioned below to load DPDK driver with UIO support: Disable Intel VT-d in system BIOS. ii. Turn off Intel iommu by adding below entry in Kernel grub/grub2 menu: intel_iommu=off iii.
  • Page 13 Chapter 4. Software/Driver Loading iii. Reboot the system for changes to take effect. iv. Load the VFIO module: [root@host~]# modprobe vfio-pci Chelsio DPDK Driver for Linux...

  • Page 14: Software/Driver Configuration And Fine Tuning

    Reserve Huge Page memory manually:  x86_64 [root@host~]# echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages  POWERPC64 [root@host~]# echo 512 > /sys/devices/system/node/node0/hugepages/hugepages- 16384kB/nr_hugepages In case of dual socket machines, run the above command for other CPU nodes. Chelsio DPDK Driver for Linux...

  • Page 15: Binding Network Ports

    Please make sure that adapter's physical function 4 is used, since it is assigned Note for all NIC functions of the adapter.  Run the following command to bind network ports to DPDK environment, with UIO support: [root@host~]# dpdk_nic_bind.py --bind=igb_uio <PCI-ID-PF4> Now, verify using: [root@host~]# dpdk_nic_bind.py –status Example: ...

  • Page 16: Unbinding Network Ports

    PCIe Gen3 x8/x16 slot.  using traffic with multiple tuples. The 'range' option of DPDK-Pktgen app can be used to create traffic with different 4-tuple values, so as to take advantage of RSS and hence, allow the traffic to be distributed across different queues/CPUs in Rx direction.

  • Page 17: Running Dpdk Test Applications

    6.1. Testpmd application The Testpmd application is provided as a part of the Chelsio DPDK driver package, and can be used to test the DPDK in a packet forwarding mode and also to access NIC hardware features such as Flow Director.
  • Page 18 Chapter 6. Running DPDK Test Applications 6.1.3. Flow Control Flow control pause TX/RX is disabled by default and can be enabled via Testpmd as follows: testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 0 testpmd>...
  • Page 19 Chapter 6. Running DPDK Test Applications  Testpmd This method involves using the mtu command, which changes the MTU of an individual port without having to stop the selected port. To configure each port individually, run the mtu command as follows: testpmd>...

  • Page 20: Pktgen Application

    Configure Pktgen tool to use single Tx and Rx queue, mapping core 1 to Rx and core 2 to Tx queue on port 0: [root@host~]# pktgen -c f -n 4 -- -T -P -m "[1:2].0" –N Pktgen created by Keith Wiles -- >>> Powered by Intel® DPDK <<< - Ports 0-3 of 4 ** Main Page ** Copyright (c) <2010-2015>, Wind River...
  • Page 21 Chapter 6. Running DPDK Test Applications Link State <UP-10000-FD> ---TotalRate--- Pkts/s 14880593 14880593 MBits/s Rx/Tx 0/9999 0/9999 Broadcast Multicast 64 Bytes 65-127 128-255 256-511 512-1023 1024-1518 Runts/Jumbos Errors Rx/Tx Total Rx Pkts Tx Pkts 175605364 Rx MBs Tx MBs 118006...
  • Page 22 Chapter 6. Running DPDK Test Applications iii. Start Tx traffic on port 0 Pktgen> start 0 iv. Stop Tx traffic on port 0 Pktgen> stop 0  Running multiple Tx and Rx queue traffic Configure Pktgen tool to use 4 Tx and Rx queues, mapping cores 1-4 to Rx and 5-8 to Tx queues, on port 0: [root@host~]# pktgen -c 3ff -n 4 -- -T -P -m "[1-4:5-8].0"...
  • Page 23 Chapter 6. Running DPDK Test Applications ii. Set packet size to 9018B on all ports Pktgen> set all size 9018 iii. Start Tx traffic on port 0 Pktgen> start 0 iv. Stop Tx traffic on port 0 Pktgen> stop 0...

  • Page 24: Classification And Filtering

    The masks will allow specifying a match for a range of values. You can create up to 496 LE-TCAM filters. Currently supported on T5 adapters only. Note 7.1. Pre-requisites To use the Classification and Filtering feature, DPDK driver should be loaded with UIO support. Software/Driver Loading for step-by-step instructions. 7.2. Usage 7.2.1. Filter Modes The Classification and Filtering feature is configured by specifying the filter modes in the firmware configuration file, t5-config.txt, located in /lib/firmware/cxgb4/...
  • Page 25 Using any other filter mode combination is strictly not supported. Important fragmentation,mpshittype,macmatch,ethertype,protocol,port fragmentation,mpshittype,macmatch,ethertype,protocol,fcoe fragmentation,mpshittype,macmatch,ethertype,tos,port fragmentation,mpshittype,macmatch,ethertype,tos,fcoe fragmentation,mpshittype,macmatch,ethertype,port,fcoe fragmentation,mpshittype,macmatch,protocol,tos,port,fcoe fragmentation,mpshittype,macmatch,protocol,vlan,fcoe fragmentation,mpshittype,macmatch,protocol,vnic_id,fcoe fragmentation,mpshittype,macmatch,tos,vlan,fcoe fragmentation,mpshittype,macmatch,tos,vnic_id,fcoe fragmentation,mpshittype,macmatch,vlan,port,fcoe fragmentation,mpshittype,macmatch,vnic_id,port,fcoe fragmentation,mpshittype,ethertype,protocol,tos,port,fcoe fragmentation,mpshittype,ethertype,vlan,port fragmentation,mpshittype,ethertype,vlan,fcoe fragmentation,mpshittype,ethertype,vnic_id,port fragmentation,mpshittype,ethertype,vnic_id,fcoe fragmentation,mpshittype,protocol,tos,vlan,port fragmentation,mpshittype,protocol,tos,vlan,fcoe fragmentation,mpshittype,protocol,tos,vnic_id,port fragmentation,mpshittype,protocol,tos,vnic_id,fcoe fragmentation,mpshittype,protocol,vlan,port,fcoe fragmentation,mpshittype,protocol,vnic_id,port,fcoe fragmentation,mpshittype,tos,vlan,port,fcoe fragmentation,mpshittype,tos,vnic_id,port,fcoe fragmentation,mpshittype,vlan,vnic_id,fcoe fragmentation,macmatch,ethertype,protocol,port,fcoe fragmentation,macmatch,ethertype,tos,port,fcoe fragmentation,macmatch,protocol,vlan,port,fcoe Chelsio DPDK Driver for Linux...
  • Page 26 : A hexadecimal bit mask of the cores to run on. Note that core numbering can change between platforms and should be determined beforehand. -n <channels> : Number of memory channels per processor socket. : Enable interactive mode. Chelsio DPDK Driver for Linux...
  • Page 27 Ingress port mask 0: Don’t care 0x7: Exact match 0x6: Two ports starting with port specified for iport fcoe_bit FCoE based Filtering 0: Disable 1: Enable fcoe_bit_mask FCoE traffic mask 0: Don’t care 1: Exact match Chelsio DPDK Driver for Linux...
  • Page 28 Ingress packets will be passed through set ingress queues. switch: Ingress packets will be routed to an output port with optional header rewrite. queue_id Rx queue ID egress_port_redirect port-none,port-redirect Egress Port redirection port-none: No redirection Chelsio DPDK Driver for Linux...
  • Page 29 Destination port that needs to be translated to. 0-495 fd_id_value Filter ID Note: The rule with the lowest filter ID takes the highest precedence. To calculate the range of possible values, please use the U32 Port Masks Calculator at http://blog.of.geek.nz/2012/07/22/u32-port-masks-calculator/ Chelsio DPDK Driver for Linux...
  • Page 30 <proto_mask> <src_ip_address> <src_ip_mask> <dst_ip_address> <dst_ip_mask> <src_port> <src_port_mask> <dst_port> <dst_port_mask> <action> queue <queue_id> <egress_port_redirect> <egress_port> <mac_addr_rewrite> <src_mac> <dst_mac> <vlan_operation> <new_vlan> <nat_operation> <nat_src_ip> <nat_dst_ip> <nat_src_port> <nat_dst_port> fd_id <fd_id_value> Packets that don’t meet the filter accept criteria will be forwarded to the DPDK Note stack. Examples ...
  • Page 31 The above filter rule will switch only fragmented packets.  drop action The above filter rule will drop all packets received on port 0.  fwd (forward) action The above filter rule will forward packets to the application using queue 0. Chelsio DPDK Driver for Linux...
  • Page 32 <ether_type> <ether_type_mask> vlan <inner_vlan> <inner_vlan_mask> <outer_vlan> <outer_vlan_mask> ip <frag_bit> <frag_bit_mask> <tos> <tos_mask> <proto> <proto_mask> <src_ip_address> <src_ip_mask> <dst_ip_address> <dst_ip_mask> <src_port> <src_port_mask> <dst_port> <dst_port_mask> <action> queue <queue_id> <egress_port_redirect> <egress_port> <mac_addr_rewrite> <src_mac> <dst_mac> <vlan_operation> <new_vlan> <nat_operation> <nat_src_ip> <nat_dst_ip> <nat_src_port> <nat_dst_port> fd_id <fd_id_value> Chelsio DPDK Driver for Linux...

  • Page 33: Layer 2 Example

    102.1.1.2/24 eth0 eth0 node1 node2  Configure IP addresses on node1 and node2 node1: [root@host~]# ifconfig eth0 102.1.1.1/24 up [root@host~]# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:07:43:04:7D:50 inet addr:102.1.1.1 Bcast:102.1.1.255 Mask:255.255.255.0 inet6 addr: fe80::207:43ff:fe04:7d50/64 Scope:Link Chelsio DPDK Driver for Linux...
  • Page 34 Create filter rule to switch all packets from port0 to port1: ii. Create another filter rule to switch all packets from port1 to port0: iii. Generate traffic from node1 to node2 and vice versa. Observe that traffic is forwarded successfully: Chelsio DPDK Driver for Linux...

  • Page 35: Forwarding Multi-Cast Traffic

    Add the multi-cast MAC address to MPS TCAM table and note the corresponding index: ii. Create a filter rule to switch packets using the index of multi-cast mac address and mac- match-type as multicast, from port0 to port1: Chelsio DPDK Driver for Linux...
  • Page 36 Chapter 7. Classification and Filtering iii. Generate traffic using the traffic generator with the multi-cast mac address as the destination mac address. Observe that node2 receives the multi-cast traffic: Chelsio DPDK Driver for Linux...

  • Page 37: Filtering Using Outer Vlan

    In the firmware configuration file, set register, filterMode and filterMask as follows: ii. Create filter to drop packets with outer VLAN ID = 75 iii. Transmit packets from peer machine with outer VLAN as 75. Observe that packets will be dropped by the DPDK machine. Chelsio DPDK Driver for Linux...

  • Page 38: Software/Driver Unloading

    Chapter 8. Software/Driver Unloading 8. Software/Driver Unloading UIO Support Run the following commands to unload DPDK driver with UIO support: [root@host~]# rmmod igb_uio [root@host~]# rmmod uio If unloading uio module reports an error, unload the following dependent modules and try again:...

  • Page 39: Software/Driver Uninstallation

    Chapter 9. Software/Driver Uninstallation 9. Software/Driver Uninstallation Change your working directory to Chelsio-DPDK-x.x.x.x directory: [root@host~]# cd Chelsio-DPDK-x.x.x.x ii. Uninstall the DPDK driver using the following command: [root@host~]# make uninstall Chelsio DPDK Driver for Linux...

  • Page 40: Software/Driver Update

    For any distribution specific problems, please check README and Release Notes included in the release for possible workaround. Please visit Chelsio support web site http://service.chelsio.com/ for regular updates on various software/drivers. You can also subscribe to our newsletter for the latest software updates. Chelsio DPDK Driver for Linux...

  • Page 41: Appendix

    TERMINATE INSTALLATION PROCESS. 1. License. Chelsio Communications, Inc. ("Chelsio") hereby grants you, the Licensee, and you hereby accept, a limited, non-exclusive, non-transferable license to install and use the Software with one or more Chelsio network adapters on a single server computer for use in communicating with one or more other computers over a network.
  • Page 42 United States and foreign law. You agree to comply with all applicable international and national laws that apply to the Software, including the U.S. Export Administration Regulations, as well as end-user, end-use, and destination restrictions issued by U.S. and other governments. Chelsio DPDK Driver for Linux...
  • Page 43 Agreement. Should you have any questions concerning this Agreement, you may contact Chelsio by writing to: Chelsio Communications, Inc. 209 North Fair Oaks Avenue, Sunnyvale, CA 94085 Chelsio DPDK Driver for Linux...

Table of Contents