Page 1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x First Published: 2015-01-31 Last Modified: 2020-08-31 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883...
Page 2 Any products and features described herein as in development or available at a future date remain in varying stages of development and will be offered on a when-and if-available basis. Any such product or feature roadmaps are subject to change at the sole discretion of Cisco and Cisco will have no liability for delay in the delivery or failure to deliver any products or feature roadmap items that may be set forth in this document.
Page 3 Sequencing of Ingress Traffic Actions Sequencing of Egress Traffic Actions High Availability Requirements for QoS Features QoS Feature Configuration with MQC QoS Statistics Default QoS Behavior Virtual Device Contexts Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 4: Table Of Contents
About QoS TCAM Carving About QoS TCAM Lite Regions Guidelines and Limitations Configuring QoS TCAM Carving Enabling Layer 3 QoS (IPv6) Enabling VLAN QoS (IPv4) Notes for Enabling VLAN QoS Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 5 Trust Boundaries Class of Behavior Prerequisites for Marking Guidelines and Limitations Configuring Marking Configuring DSCP Marking Configuring IP Precedence Marking Configuring CoS Marking Configuring CoS Marking for FEX Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 6 Guidelines and Limitations Configuring Queuing and Scheduling Configuring Type Queuing Policies Configuring Congestion Avoidance Configuring Tail Drop on Egress Queues Configuring WRED on Egress Queues Configuring AFD on Egress Queues Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 7 Information About Link Level Flow Control Link Level Flow Control on Interfaces Link Level Flow Control on Ports Mismatched Link Level Flow Control Configurations How to Configure Link Level Flow Control Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 8 Configuration Examples For Monitoring QoS Statistics C H A P T E R 1 3 Micro-Burst Monitoring Micro-Burst Monitoring Guidelines and Limitations for Micro-Burst Monitoring Configuring Micro-Burst Detection Per-Queue Clearing Micro-Burst Detection Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x viii...
Page 9 FEX QoS Configuration Information TCAM Carving for FEX QoS FEX QoS Configuration Example Verifying the FEX QoS Configuration A P P E N D I X B Additional References RFCs Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 10 Contents Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 11 • Documentation Feedback, on page xii • Communications, Services, and Additional Information, on page xii Audience This publication is for network administrators who install, configure, and maintain Cisco Nexus switches. Document Conventions Command descriptions use the following conventions: Convention Description...
Page 12 An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line. Related Documentation for Cisco Nexus 9000 Series Switches The entire Cisco Nexus 9000 Series switch documentation set is available at the following URL: http://www.cisco.com/en/US/products/ps13386/tsd_products_support_series_home.html Documentation Feedback To provide technical feedback on this document, or to report an error or omission, please send your comments to nexus9k-docfeedback@cisco.com.
Page 13 Cisco Bug Search Tool Cisco Bug Search Tool (BST) is a web-based tool that acts as a gateway to the Cisco bug tracking system that maintains a comprehensive list of defects and vulnerabilities in Cisco products and software. BST provides you with detailed defect information about your products and software.
Page 14 Preface Preface Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 15 New and Changed Information, on page 1 New and Changed Information This table summarizes the new and changed features for the Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide and where they are documented. Table 1: New and Changed Features...
Page 16 9300-FX2, and Cisco Nexus page 173 N9K-C9364C switches. VLAN QoS Added support for VLAN QoS 7.0(3)F3(3) Notes for Enabling VLAN QoS, for Cisco Nexus 9508 switch. on page 49 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 17 Active Queue Management (AQM) algorithm to avoid traffic congestion. DCBXP Added support for the Cisco 7.0(3)I5(1) About Priority Flow Control, Nexus 9200 and 9300-EX on page 149 Series switches. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 18 Changed the behavior of the no 7.0(3)I4(2) Guidelines and Limitations for lldp tlv-select dcbxp command Priority Flow Control such that PFC mode will be disabled for interfaces on both sides of back-to-back switches. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 19 Enabling Egress QoS (IPv4) TCAM carving. Egress policing support Added support to apply the 7.0(3)I1(1) Configuring Egress Policing policing instructions in a QoS policy map to egress packets. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 20 New and Changed Information New and Changed Information Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 21 Default QoS Behavior, on page 11 • Virtual Device Contexts, on page 11 Licensing Requirements For a complete explanation of Cisco NX-OS licensing recommendations and how to obtain and apply licenses, see the Cisco NX-OS Licensing Guide. About QoS Features You use the QoS features to provide the most desirable flow of traffic through a network.
Page 22 COS, IP precedence and DSCP, and internal labels (such as QoS groups) that can be used in subsequent actions. Marking QoS groups is used to identify the traffic type for queuing and scheduling traffic. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 23 The system performs actions for QoS policies only if you define them under the type qos service policies. Sequencing of Ingress Traffic Actions The sequence of QoS actions on ingress traffic is as follows: 1. Classification 2. Marking 3. Policing Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 24 1. Queuing and scheduling High Availability Requirements for QoS Features The Cisco NX-OS QoS software recovers its previous state after a software restart, and it is capable of a switchover from the active supervisor to the standby supervisor without a loss of state.
Page 25 Virtual Device Contexts Cisco NX-OS can segment operating system and hardware resources into virtual device contexts (VDCs) that emulate virtual devices. The Cisco Nexus 9000 Series device currently does not support multiple VDCs. All device resources are managed in the default VDC.
Page 26 Overview Virtual Device Contexts Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 27: Using Modular Qos Cli
• Session Manager Support for QoS, on page 38 About MQC Cisco Modular Quality of Service Command Line Interface (MQC) provides a language to define QoS policies. You configure QoS policies by following these three steps: 1. Define traffic classes.
Page 28: System Classes
4q mode or system-defined objects for 8q mode. On the Cisco Nexus device, a system class is uniquely identified by a qos-group value. A total of four system classes are supported. The device supports one default class which is always present on the device. Up to three additional system classes can be created by the administrator.
Page 29: Type Qos Policies
You use type queuing policies to shape and queue packets. The following figure shows the QoS policy structure with associated MQC objects of type queuing. The MQC objects are shown in bold. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 30: System-Defined Mqc Objects
• Cisco Nexus 9300-FX switches • Cisco Nexus 9300-FX2 switches • Cisco Nexus 9300-GX switches • Cisco Nexus 9504, 9508, and 9516 switches with -EX or -FX line cards. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 31: System-Defined Mqc Objects For 4Q Mode
When you configure QoS features, and the system requests MQC objects, you can use the following system-defined objects: Note The Cisco Nexus 9000 series NX-OS system operates in 4q mode by default. System-defined MQC objects for 4q mode are the default MQC objects. Note System-defined MQC objects for 4q mode are not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)).
Page 32 1 class type queuing c-out-q2 bandwidth remaining percent 0 class type queuing c-out-q1 bandwidth remaining percent 0 class type queuing c-out-q-default bandwidth remaining percent 100 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 33: System-Defined Mqc Objects For 8Q Mode
Class Map Queue Name Description c-out-8q-q-default Egress default queue — QoS group 0 c-out-8q-q1 Egress queue 1 — QoS group 1 c-out-8q-q2 Egress queue 2 — QoS group 2 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 34 Network-qos class — QoS group 1 c-8q-nq2 Network-qos class — QoS group 2 c-8q-nq3 Network-qos class — QoS group 3 c-8q-nq4 Network-qos class — QoS group 4 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 35 0 class type queuing c-out-8q-q2 bandwidth remaining percent 0 class type queuing c-out-8q-q1 bandwidth remaining percent 0 class type queuing c-out-8q-q-default bandwidth remaining percent 100 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 36: Changing To 8Q Mode
1500 Changing to 8q Mode Note The Cisco Nexus 9000 series NX-OS system operates in 4q mode by default. Use the following guidelines to change to 8q mode: • Change the network-qos policy to 8q mode. You can either activate the default-8q-nq-policy (which is the system created 8q default network-qos policy);...
Page 37 ERROR: set on qos-group 4-7 is supported only on 8q capable platforms Example of Changing to 8q Mode The following is an example of changing to 8q mode: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 38 Using Modular QoS CLI Example of Changing to 8q Mode Note This example is not applicable to the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). switch# qos copy policy-map type network-qos default-8q-nq-policy prefix my switch# show policy-map type network-qos Type network-qos policy-maps...
Page 39 0 class type queuing c-out-8q-q3 bandwidth remaining percent 0 class type queuing c-out-8q-q2 bandwidth remaining percent 0 class type queuing c-out-8q-q1 bandwidth remaining percent 0 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 40 (match-any) priority level 1 Class-map (queuing): c-out-8q-q6 (match-any) bandwidth remaining percent 10 Class-map (queuing): c-out-8q-q5 (match-any) bandwidth remaining percent 10 Class-map (queuing): c-out-8q-q4 (match-any) bandwidth remaining percent 10 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 41 Ethernet2/1 Service-policy (qos) input: SNMP Policy Index: 285226505 Class-map (qos): c1 (match-all) Match: dscp 10 set qos-group 1 Class-map (qos): c2 (match-all) Match: dscp 20 set qos-group 4 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 42: Changing From 8Q Mode To 4Q Mode
Changing from 8q Mode to 4q Mode Note Changing from 8q mode to 4q mode is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). Use the following guidelines to change from 8q mode to 4q mode: • Ensure that none of the active input QoS policies have set qos-group action for qos-groups 4-7, so that no traffic flows towards queues 4-7.
Page 43 (Optional) Displays information about all configured class maps, all class maps of type queuing, or a selected class Example: map of type queuing. switch(config)# show class-map type queuing Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 44: Configuring Or Modifying A Policy Map
40 characters. switch(config-pmap-que)# Step 5 exit Exits policy map mode and enters global configuration mode. Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 45: Applying Descriptions To Mqc Objects
Specify the MQC object whose description you want to set: • Class-map: • Class-map: Creates or accesses the class map and then enters class-map mode. The class-map name can contain Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 46: Verifying An Mqc Object
Displays information about all configured policy maps, all policy [policy-map-name | default-out-policy]] maps of type queuing, a selected policy map of type queuing, or the default output queuing policy. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 47: Attaching And Detaching A Qos Policy Action
• Present or not—Interface where a policy is either attached or not, but not applied. Table 13: QoS Policy Interfaces Port Policy Port-Channel Policy VLAN Policy Applied Not present Present or not Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 48: Configuring A Service Policy For A Layer 2 Interface
Specifies the policy map to use as the service policy for the output | queuing output} policy-map-name [no-stats] Layer 2 interface. There are two policy-map configuration modes: Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 49: Configuring A Service Policy For A Layer 3 Interface
5. show policy-map interface interface slot/port type {qos | queuing} 6. copy running-config startup-config DETAILED STEPS Command or Action Purpose Step 1 configure terminal Enters global configuration mode. Example: switch# configure terminal switch(config)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 50: Attaching The System Service Policy
The service-policy command specifies the system class policy map as the service policy for the system. SUMMARY STEPS 1. configure terminal 2. system qos 3. service-policy type {network-qos | queuing output} policy-map-name Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 51: Attaching A Qos Policy Action To A Vlan
3. service-policy [type qos] {input} | {qos output } {policy-map-name} [no-stats] 4. show policy-map [interface interface | vlan vlan-id] [input] [type qos | queuing] [class [type qos | queuing] class-map-name] 5. copy running-config startup-config Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 52: Session Manager Support For Qos
Session Manger supports the configuration of QoS. This feature allows you to verify the QoS configuration and confirm that the resources required by the configuration are available prior to committing them to the running configuration. For information about Session Manager, see the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide.
Page 53: Configuring Qos Tcam Carving
256 entries actually consumes 512 physical TCAM entries. On Cisco Nexus 9200 Series switches, the egress TCAM size is 2K, and the ingress TCAM size is 4K. The concepts of TCAM slices and single- and double-wide regions do not apply to these switches. For example, the ing-ifacl region can host IPv4, IPv6, or MAC type entries.
Page 54 About QoS TCAM Carving Note In addition to the above TCAM, for ALE enabled devices, a separate TCAM in the Cisco Nexus C9396PX (uplink ports) and Cisco Nexus C93128TX (uplink ports) ASIC is used for the QoS classification policies applied on 40G uplink ports. By default, this separate TCAM is carved for Layer 3 QoS (IPV4), Layer 2 Port QoS (IPV4), and VLAN QoS (IPV4) with 256 entries each.
Page 55 40G uplink ports. Note ** The region is applicable only for 100G enabled devices (such as the Cisco Nexus 9300 platform switch with the N9K-M4PC-CFP2 GEM or the Cisco Nexus 9500 platform switch with the Cisco Nexus 9408PC-CFP2 line card) and are required for classification policies and QoS scheduling applied on 100G uplink ports.
Page 56: About Qos Tcam Lite Regions
Either the regular version or the lite version of the QOS TCAM can be enabled. Both cannot be enabled at the same time. For example, either the IPv4 Port QoS or the IPv4 Port QoS lite version can be enabled at any one time. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 57: Guidelines And Limitations
T2 QoS region, for example, qos and l3-qos regions. This command removes the TCAM restrictions that are associated with the QoS classifications on the ALE ports and it is only supported on Cisco Nexus 9000 Series switches with Application Leaf Engine (ALE).
Page 58: Configuring Qos Tcam Carving
512 entries). • On the Cisco Nexus 9508 switch with the -R series line card, VLAN QoS is only supported with Cisco NX-OS Release 7.0(3)F3(3) and later releases.
Page 59 Configuring QoS TCAM Carving Enabling Layer 3 QoS (IPv6) Table 19: Default TCAM Region Configuration (Ingress) for the Cisco Nexus 9504, Cisco Nexus 9508, and Cisco Nexus 9516 devices Region Name Size Width Total Size IPV4 RACL 1536 1536 L3 QoS(IPV4)
Page 60 Warning: Please reload the linecard for the configuration to take effect switch(config)# hardware access-list tcam region ipv6-l3qos 256 Warning: Please reload the linecard for the configuration to take effect Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 61: Enabling Vlan Qos (Ipv4)
The following table list the default sizes for the ingress TCAM regions for ALE enabled devices. Table 23: Default TCAM Region Configuration (Ingress) Region Name Size Width Total Size PACL (IPV4) Port QoS (IPV4) VACL (IPV4) RACL(IPV4) System COPP Redirect SPAN VPC Converg Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 62 Table 24: Updated TCAM Region Configuration After Reducing the IPv4 Port QoS Ingress Region Name Size Width Total Size PACL (IPV4) Port QoS (IPV4) VLAN QoS(IPV4) VACL (IPV4) RACL(IPV4) System COPP Redirect SPAN VPC Converg Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 63: Notes For Enabling Vlan Qos
Enabling FEX QoS (IPv4) Note The FEX QoS feature is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). To enable FEX QoS (IPv4), you must decrease the TCAM size of another region and then increase the TCAM size to enable the new FEX QoS (IPv4) region.
Page 64: Enabling Egress Qos (Ipv4)
TCAM size to enable the newQoS (IPv4) TCAM region. Note The egress QoS feature is not supported on the Cisco Nexus 9508 switch (Cisco NX-OS 7.0(3)F3(3)). Note Egress marking and policing is supported on all Network Forwarding Engine (NFE) platforms. Egress classification for egress packet scheduling is supported only on 100G platforms.
Page 65 Example: single wide. switch(config)# hardware access-list tcam region egr-l3-vlan-qos 256 Warning: Please reload all linecards for the configuration to take effect switch(config)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 66: Using Templates To Configure Tcam Region Sizes
Using Templates to Configure TCAM Region Sizes Note Using templates to configure TCAM region sizes is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). Beginning with Cisco NX-OS Release 7.0(3)I3(1), you can use create and apply custom templates to configure TCAM region sizes.
Page 67: Verifying Qos Tcam Carving
To display the configuration of a TCAM template, use the show hardware access-list tcam template {all | nfe | nfe2 | l2-l3 | l3 | template-name} command where: • all—Displays configuration for all TCAM templates. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 68 9500 Series, 3164Q, and 31128PQ devices. • nfe2—The default TCAM template for NFE2-enabled Cisco Nexus 9500, 3232C, and 3264Q devices. • l2-l3—The default TCAM template for Layer 2-to-Layer 3 configurations on Cisco Nexus 9200 Series switches. • l3—The default TCAM template for Layer 3 configurations on Cisco Nexus 9200 Series switches.
Page 69: Configuring Classification
You can specify multiple match criteria, you can choose to not match on a particular criterion, or you can determine the traffic class by matching any or all criteria. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 70: Prerequisites For Classification
Guidelines and Limitations Classification has the following configuration guidelines and limitations: • Beginning with Cisco Nexus NX-OS Release 7.0(3)I7(9), the Cisco Nexus 9504 switch with the Cisco Nexus 9464PX or 9464TX line card supports DSCP wildcard mask. • The show commands with the internal keyword are not supported.
Page 71 If both are present, the device acts on the port policy and ignores the VLAN policy. • When a Cisco Nexus Fabric Extender (FEX) is connected and in use, data traffic should not be marked with a CoS value of 7. CoS 7 is reserved for control traffic transiting the Fabric Extender.
Page 72: Configuring Traffic Classes
1/1 switch(config-if)# switchport access vlan 20 switch(config-if)# switchport voice vlan dot1p • Cisco Nexus 9504 and Cisco Nexus 9508 switches with the following line cards do not support QoS match acl with fragments: • Cisco Nexus 96136YC-R •...
Page 73: Examples: Configuring Acl Classification
Adding an explicit DENY ALL at the end of a QoS matching ACL causes the QoS ACL to permit all traffic. The following examples, D1 and E1, produce the same QoS matching results: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 74: Configuring A Dscp Wildcard Mask
With this granularity, you can treat these flows by policing them to protest the rest of the traffic, or assign them to a qos-group for further QoS operations. Note Only Cisco Nexus 9504 switches with the Cisco Nexus 9464PX or 9464TX line card support the DSCP wildcard mask feature. SUMMARY STEPS 1.
Page 75 10 permit ip 10.1.1.1/24 20.1.1.2/24 dscp 33 dscp-mask 30 switch(config-acl)# exit switch(config)# class-map type qos match-any class_dscp_mask switch(config-cmap-qos)# match access-list acl-01 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 76: Configuring Dscp Classification
CS7 (precedence 7) dscp (111000)—decimal value 56 default Default dscp (000000)—decimal value 0 EF dscp (101110)—decimal value 46 SUMMARY STEPS 1. configure terminal 2. class-map [type qos] [match-any | match-all] class-name Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 77: Configuring Ip Precedence Classification
You can classify traffic based on the precedence value in the type of service (ToS) byte field of the IP header. The precedence values are listed in the following: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 78 Use the not keyword to match on values that do not match switch(config-cmap-qos)# match precedence 1-2, 5-7 the specified range. Step 4 exit Exits global class-map queuing mode and enters global configuration mode. Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 79: Configuring Protocol Classification
3. match [not] protocol {arp | bridging | cdp | dhcp | isis} 4. exit 5. copy running-config startup-config DETAILED STEPS Command or Action Purpose Step 1 configure terminal Enters global configuration mode. Example: switch# configure terminal switch(config)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 80: Configuring Layer 3 Packet Length Classification
This feature is designed for IP packets only. SUMMARY STEPS 1. configure terminal 2. class-map [type qos] [match-any | match-all] class-name 3. match [not] packet length packet-length-list 4. exit 5. copy running-config startup-config Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 81: Configuring Cos Classification
ID Tag field and is referred to as user_priority. SUMMARY STEPS 1. configure terminal 2. class-map [type qos] [match-any | match-all] class-name 3. match [not] cos cos-list 4. exit 5. copy running-config startup-config Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 82: Configuring Cos Classification For Fex
Configuring CoS Classification for FEX Note The CoS Classification for FEX feature is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). You can classify traffic based on the class of service (CoS) for a FEX. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 83 (Optional) Saves the running configuration to the startup copy running-config startup-config configuration. Example: switch(config)# copy running-config startup-config Example This example shows how to configure the CoS class-map configuration: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 84: Configuring Ip Rtp Classification
RTP. Values can range from switch(config-cmap-qos)# match ip rtp 2000-2100, 2000 to 65535. Use the not keyword to match on values 4000-4100 that do not match the specified range. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 85: Verifying The Classification Configuration
Configuration Examples for Classification The following example shows how to configure classification for two classes of traffic: class-map class_dscp match dscp af21, af32 exit class-map class_cos match cos 4, 5-6 exit Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 86 Configuring Classification Configuration Examples for Classification Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 87: Configuring Marking
Locally significant QoS values that can be manipulated and matched within the system. The range is from 0 to 3. Ingress Status of the marking applies to incoming packets. Layer 2 VLAN ID Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 88: Trust Boundaries
Copied from 3 MSB of ToS Bridged multicast with Layer 3 state for group Copied from 3 MSB of ToS Bridged multicast with no Layer 3 state for group Unchanged Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 89: Prerequisites For Marking
Configuring Marking Prerequisites for Marking Note CoS behavior per traffic type is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). Prerequisites for Marking Classification has the following prerequisites: • You must be familiar with using modular QoS CLI.
Page 90: Configuring Marking
• Span traffic automatically gets classified into qos-group 9 and is scheduled at absolute low priority. • Egress QoS policies are not supported on Cisco Nexus 9200 platform switches (For 7.0(3)I3(1)). • QOS marking policies can be enabled on subinterfaces Configuring Marking You can combine one or more of the marking features in a policy map to control the setting of QoS values.
Page 91 3. class [type qos] {class-name | class-default} [insert-before before-class-name] 4. set dscp dscp-value DETAILED STEPS Command or Action Purpose Step 1 Enters global configuration mode. configure terminal Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 92: Configuring Ip Precedence Marking
Table 32: Precedence Values Value List of Precedence Values IP precedence value critical Critical precedence (5) flash Flash precedence (3) flash-override Flash override precedence (4) immediate Immediate precedence (2) Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 93 0 to 7. You can enter one of the values Example: shown in the above Precedence Values table. switch(config-pmap-c-qos)# set precedence 3 Example This example shows how to display the policy-map configuration: switch# show policy-map policy1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 94: Configuring Cos Marking
Sets the CoS value to cos-value. The value can range from 0 to 7. Example: switch(config-pmap-c-qos)# set cos 3 switch(config-pmap-c-qos)# Example This example shows how to display the policy-map configuration: switch# show policy-map policy1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 95: Configuring Cos Marking For Fex
Configuring CoS Marking for FEX Note The CoS Marking for FEX feature is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). You can mark traffic based on the class of service (CoS) for a FEX. Before you begin Before configuring the FEX, enable feature-set fex.
Page 96: Configuring Dscp Port Marking
DSCP value is preserved. Note • You can attach only one policy type qos map to each interface in each direction. • The DSCP value is trust on the Layer 3 port of a Cisco NX-OS device. SUMMARY STEPS configure terminal...
Page 97 Standard DSCP Values table in the Configuring Example: DSCP Marking section. switch(config-pmap-c-qos)# set dscp af22 switch(config-pmap-c-qos)# Step 11 Returns to policy-map configuration mode. exit Example: switch(config-pmap-c-qos)# exit switch(config-pmap-qos)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 98: Verifying The Marking Configuration
Configuration Examples for Marking The following example shows how to configure marking: configure terminal policy-map type qos untrust_dcsp class class-default set precedence 3 set qos-qroup 3 set dscp 0 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 99: Configuring Policing
For more information about policers, see RFC 2697 and RFC 2698. Shared Policers Note The shared policer feature is only supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3) and later 7.0(3)F3(x) releases). Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 100: Prerequisites For Policing
• Cisco Nexus 9636C-RX • Cisco Nexus 96136YC-R • The egress RACL and egress QoS features are not supported on the Cisco Nexus 9508 switch (Cisco NX-OS Release 7.0(3)F3(3). • Egress QoS policy statistics for CPU generated traffic are not supported on the following: •...
Page 101 They only support the drop action for violate in the egress direction. • VLAN Egress QOS and Egress Qos on L2PO are not supported on Cisco Nexus 97160YC-EX, 9732C-EX, 9736C-EX line cards.
Page 102: Configuring Policing
• Egress QoS policies are not supported on Cisco Nexus 9200 platform switches (For 7.0(3)I3(1)). • Cisco Nexus 9200 platform switches support a 1-rate 2-color policer. A 2-rate 3-color policer is not supported on Cisco Nexus 9200 platform switches (For Cisco NX-OS Relese 7.0(3)I3(1)).
Page 103 Data Rates for the violate {drop | set-cos-transmit | set-dscp-transmit | set-prec-transmit | set-qos-transmit }]]} police Command table and the Burst Sizes for the police Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 104: Configuring 1-Rate And 2-Rate, 2-Color And 3-Color Policing
The type of policer created by the device is based on a combination of the police command arguments described in the following Arguments to the police Command table. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 105 In this section, cir indicates its value but not necessarily the keyword itself. The combination of these arguments and the resulting policer types and actions are shown in the following Policer Types and Actions from Police Arguments Present table. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 106 The policer actions that you can specify are described in the following Policer Actions for Exceed or Violate table and the following Policer Actions for Conform table. Note Only drop and transmit actions are supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3) and later). Table 35: Policer Actions for Exceed or Violate...
Page 107 [exceed {drop} [violate {drop | set-cos-transmit | set-dscp-transmit | set-prec-transmit | set-qos-transmit}]]} 5. [ violate {drop | set-cos-transmit | set-dscp-transmit | set-prec-transmit | set-qos-transmit}] 6. exit 7. exit Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 108 Exits policy-map class configuration mode and enters policy-map mode. Example: switch(config-pmap-c-qos)# exit switch(config-pmap-qos)# Step 7 Exits policy-map mode and enters global configuration exit mode. Example: switch(config-pmap-qos)# exit switch(config)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 109: Configuring Markdown Policing
5. exit 6. exit 7. show policy-map [type qos] [policy-map-name] 8. copy running-config startup-config DETAILED STEPS Command or Action Purpose Step 1 configure terminal Enters global configuration mode. Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 110 Example: switch(config)# show policy-map Step 8 copy running-config startup-config (Optional) Saves the running configuration to the startup configuration. Example: switch(config)# copy running-config startup-config Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 111: Configuring Shared Policers
Cisco documentation. Note The shared policer feature is only supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3) and later). Note When the shared policer is applied on interfaces or VLANs with member ports that are across different cores or instances, the rate becomes two times the configured cir rate.
Page 112 Displays information about all configured policy maps or [policy-map-name | qos-dynamic] a selected policy map of type qos. Step 9 (Optional) switch(config)# copy running-config Saves the running configuration to the startup configuration. startup-config Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 113: Verifying The Policing Configuration
10 mbps pir 20 mbps conform transmit exceed set dscp dscp table cir-markdown-map violate drop policy-map type qos udp_policy class type qos udp_qos police aggregate udp_10mbps Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 114 Configuring Policing Configuration Examples for Policing Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 115: Configuring Queuing And Scheduling
The queuing and scheduling processes allow you to control the bandwidth that is allocated to the traffic classes so that you achieve the desired trade-off between throughput and latency for your network. Modifying Class Maps System-defined queuing class maps are provided. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 116: Congestion Avoidance
Note The ECN feature is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). Note Enabling WRED and ECN on a class on a network-qos policy implies that WRED and ECN is enabled for all ports in the system.
Page 117: Approximate Fair Drop
Approximate Fair Drop Approximate Fair Drop Note Approximate Fair Drop is not supported on the Cisco Nexus 9508 switch (Cisco NX-OS Release NX-OS 7.0(3)F3(3)). Approximate Fair Drop (AFD) is an Active Queue Management (AQM) algorithm that acts on long lived large flows (elephant flows) in the case of congestion, and does not impact short flows (mice flows).
Page 118 1a006400 1 <<<slice 0 Eth1/52 1a006600 1 <<<slice 1 Eth1/53 1a006800 1 <<<slice 0 switch(config)# interface ethernet 1/50 switch(config-if)# service-policy type queuing output LM-out-40G switch(config)# interface ethernet 1/51 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 119: Traffic Shaping
You can shape traffic that adheres to a particular profile to meet downstream requirements. Traffic shaping eliminates bottlenecks in topologies with data-rate mismatches. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 120: Prerequisites For Queuing And Scheduling
• Traffic shaping might increase the latency of packets due to queuing because it falls back to store-and-forward mode when packets are queued. • Traffic shaping is not supported on the Cisco Nexus 9300 ALE 40G ports. For more information on ALE 40G uplink ports, see the Limitations for ALE 40G Uplink Ports on the Cisco Nexus 9000 Series Switches.
Page 121 • The FEX QoS system level queuing policy does not support multiple priority levels. • Assigning a high alpha value on a Cisco Nexus 9200 platform switch uses more than the expected 50% of the available buffer space. Assigning a lower alpha value (7 or less) assures the usage of the expected 50% of the available buffer space.
Page 122 Generally, Cisco recommends not to disable the buffer-boost feature. However, disabling the buffer-boost is necessary when there is a need to port channel two different member ports from Cisco Nexus 9636PQ based line cards and Cisco Nexus 9564PX based line cards. However, Cisco does not recommend to port channel such a configuration between ACI capable leaf line cards and standalone line cards.
Page 123: Configuring Queuing And Scheduling
• Ingress queuing is not supported on devices with 100G ports. • The ingress queuing policy is supported only at the system level (and not at the interface level) for Cisco Nexus 9508 switches with the Cisco Nexus 9732C-EX line card and Cisco Nexus 93108TC-EX and 93180YC-EX switches.
Page 124: Configuring Type Queuing Policies
Assigns a weight to the class. The class will receive the bandwidth percent percentage assigned percentage of interface bandwidth if there are no strict-priority queues. If there are strict-priority queues, Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 125: Configuring Congestion Avoidance
You can configure tail drop on egress queues by setting thresholds. The device drops any packets that exceed the thresholds. You can specify a threshold based on the queue size or buffer memory that is used by the queue. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 126 The valid values for byte-based queue size are from 1 to 83886080. The valid values for dynamic queue size are from 0 to 10 as follows: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 127 Leaf Spine Engine (LSE) enabled switches. Note Setting the threshold on ALE enabled devices is only supported for the system level. It is not supported for the port level. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 128: Configuring Wred On Egress Queues
5. (Optional) Repeat Steps 3 and 4 to configure WRED for other queuing classes. 6. (Optional) congestion-control random-detect forward-nonecn DETAILED STEPS Command or Action Purpose Step 1 configure terminal Enters global configuration mode. Example: switch# configure terminal switch(config)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 129 This is a global CLI command. Allows non-ECN-capable forward-nonecn traffic to bypass WRED thresholds and grow until the egress queue-limit and tail drops. This command is intended to be Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 130: Configuring Afd On Egress Queues
AFD can be configured for an egress queuing policy. Note Approximate Fair Drop is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)). Note AFD and WRED cannot be applied at the same time. Only one can be used in a system.
Page 131 600 kbytes • Configuring AFD with ECN switch(config)# policy-map type queuing afd-ecn_8q-out switch(config-pmap-que)# class type queuing c-out-8q-q3 switch(config-pmap-c-que)# afd queue-desired 150 kbytes ecn Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 132: Configuring Congestion Management
5. (Optional) Repeat Steps 3 and 4 to assign tail drop thresholds for other queue classes. 6. exit 7. show policy-map [type queuing [policy-map-name | default-out-policy]] Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 133 (Optional) Repeat Steps 3 and 4 to assign tail drop thresholds for other queue classes. Step 6 exit Exits policy-map queue mode and enters global configuration mode. Example: switch(config-cmap-que)# exit switch(config)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 134: Configuring Bandwidth And Bandwidth Remaining For Fex
5. (Optional) Repeat Steps 3 and 4 to assign tail drop thresholds for other queue classes. 6. exit 7. show policy-map [type queuing [policy-map-name | default-out-policy]] 8. copy running-config startup-config Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 135 (Optional) Displays information about all configured policy maps, all policy maps of type queuing, a selected policy default-out-policy]] map of type queuing, or the default output queuing policy. Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 136: Configuring Priority
SPQs, you have to configure the priority on QoS group 3 and on QoS group 2. SUMMARY STEPS configure terminal policy-map type queuing {[match-first] policy-map-name} Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 137 (Optional) Assigns the percent of the bandwidth that remains to this queue. The range is from 0 to 100. Example: switch(config-pmap-c-que)# bandwidth remaining percent 25 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 138: Configuring Priority For Fex
Configuring Priority for FEX Note Priority for FEX is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3). If you do not specify the priority, the system-defined egress pq queues behave as normal queues. For information on the system-defined type queuing class maps, see the "Using Modular QoS CLI" section.
Page 139 (Optional) Configures the class map of type queuing and then enters policy-map class queuing mode. Class queuing Example: names are listed in the previous System-Defined Type switch(config-pmap-que)# class type queuing queuing Class Maps table. c-out-q3 switch(config-pmap-c-que)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 140: Configuring Traffic Shaping
40 Configuring Traffic Shaping You can configure traffic shaping on an egress queue to impose a minimum and maximum rate on it. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 141 Note Traffic shaping is not supported on the Cisco Nexus 9300 40G ports. Note The lowest value that the egress shaper can manage, per queue, is 100 Mbps on Cisco Nexus 9200 series, 9300-EX/FX/FX2, and 9700-EX/FX switches. Before you begin Configure random detection minimum and maximum thresholds for packets.
Page 142: Applying A Queuing Policy On A System
Applying a Queuing Policy on a System You apply a queuing policy globally on a system. SUMMARY STEPS 1. configure terminal 2. system qos Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 143: Verifying The Queuing And Scheduling Configuration
Controlling the QoS Shared Buffer The QoS buffer provides support per port/queue and shared space. You can control the QoS buffer that is shared by all flows by disabling or restricting reservations. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 144: Managing Dynamic Buffer Sharing
Monitoring the QoS Packet Buffer The Cisco Nexus 9000 Series device has a 12-MB buffer memory that divides into a dedicated per port and dynamic shared memory. Each front-panel port has four unicast queues and four multicast queues in egress.
Page 145 1a080600 4 Eth2/5 1a080800 4 Eth2/6 1a080a00 4 Eth2/7 1a080c00 4 Eth2/8 1a080e00 4 Eth2/9 1a081000 4 Eth2/10 1a081200 4 Eth2/11 1a081400 4 Eth2/12 1a081600 4 Eth2/13 1a081800 4 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 146: Configuration Examples For Queuing And Scheduling
100 pps max 500 pps class type queuing c-out-q2 shape min 200 pps max 1000 pps show policy-map type queuing pqu Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 147: Configuring Network Qos
• show commands with the internal keyword are not supported. • Changing the network QoS policy is a disruptive operation, and it can cause traffic drops on any or all ports. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 148: Dynamic Packet Prioritization
MTU adjustments are required under the network QoS policy to support jumbo MTU. • Network QoS is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3). • Beginning with NX-OS 7.0(3)I7(4), you can enable a network QoS pause configuration per QoS class with the pause pfc-cos cos-list receive command for the receive-only PFC option.
Page 149: Configuring Network Qos Policies
Example: switch# show policy-map type network-qos my_nq Configuring a User-Defined Network QoS Policy SUMMARY STEPS 1. configure terminal 2. class-map type network-qos match-any class-name 3. match qos-group group Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 150: Applying A Network Qos Policy On A System
Applying a Network QoS Policy on a System You apply a network QoS policy globally on a system. Applying a network QoS policy also automatically applies the corresponding queuing policies. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 151: Verifying The Network Qos
Displays the type network-qos class maps. Displays the type network-qos policy maps. show policy-map type network-qos show policy-map system type network-qos Displays the active type network-qos class maps. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 152 Configuring Network QoS Verifying the Network QoS Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 153: C H A P T E
Guidelines and Restrictions for Link Level Flow Control • show commands with the internal keyword are not supported. • Beginning with Cisco NX-OS Release 7.0(3)I7(8), syslog messages are generated at 5 minute intervals for accounting all the incoming global and link level pause frames.
Page 154: Information About Link Level Flow Control
MMU of the system may not get the expected treatment. • Beginning with NX-OS 7.0(3)I1(2), link level flow control and PFC are supported on Cisco Nexus 9300 platform switches and line cards that contain the ALE (Application Leaf Engine).
Page 155: Link Level Flow Control On Ports
2. interface ethernet 1/1 3. flowcontrol receive on 4. exit DETAILED STEPS Command or Action Purpose Step 1 configure terminal Enters global configuration mode. Example: Device# configure terminal Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 156: Configuring Link Level Flow Control Transmit
1. configure terminal 2. interface ethernet 1/1 3. flowcontrol send on 4. exit DETAILED STEPS Command or Action Purpose Step 1 configure terminal Enters global configuration mode. Example: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 157: Configuring A Link Level Flow Control Watchdog Interval
Before you begin Consider the following before configuring the Link Level Flow Control Watchdog Interval: • Link Level Flow Control Watchdog is supported on the following Cisco Nexus 9000 Series platform switches and line cards: • N9K-C9232C •...
Page 158 Configures a value for the auto-restore multiplier. The range is from 0 to 100. Example: Note The auto-restore multiplier should never be switch(config)# priority-flow-control auto-restore configured to 0. multiplier 50 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 159 LLFC watchdog auto-restore multiplier : 10 LLFC watchdog fixed-restore multiplier +----------------------------------------------------+ +----------------------------------------------------+ Ethernet1/1 Interface LLFC watchdog: [Enabled] +----------------------------------------------------+ +----------------------------------------------------+ | QOS GROUP 6 [Active] LLFC [YES] LLFC-COS [6] +----------------------------------------------------+ Stats +----------------------------------------------------+ Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 160: Configuration Examples For Link Level Flow Control
Add the queuing policy that guarantees the bandwidth for qos-group 1 and apply that under system-qos as outlined in the following example: policy-map type queuing my_queuing_policy class type queuing c-out-q-default bandwidth percent 1 class type queuing c-out-q3 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 161: Example: Configuring Link Level Flow Control Receive And Send
(Refer to the Configuring a No-Drop Policy example for information about configuring the no-drop class.) Device# configure terminal Device(config)# interface ethernet 1/1 Device(config-if)# flowcontrol receive on Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 162 (Refer to the Configuring a No-Drop Policy example for information about configuring the no-drop class.) Device# configure terminal Device(config)# interface ethernet 1/1 Device(config-if)# flowcontrol send on Device(config-if)# exit Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 163: Configuring Priority Flow Control
0 indicates a resume frame to restart the paused traffic. Note Only certain classes of service of traffic can be flow controlled while other classes are allowed to operate normally. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 164: Prerequisites For Priority Flow Control
This pause frame is a one-hop frame that is not forwarded when received by the peer. When the congestion is mitigated, PFC can request the peer to restart transmitting frames. Note Cisco Nexus 9000 Series switches support the transport of RDMA over Converged Ethernet (RoCE) v1 and v2 protocols. Prerequisites for Priority Flow Control PFC has the following prerequisites: •...
Page 165 • When a no-drop class is classified based on 802.1p CoS x and assigned a internal priority value (qos-group) of y, Cisco recommends that you use the internal priority value x to classify traffic on 802.1p CoS only, and not on any other field. The packet priority assigned is x if the classification is not based on CoS, which results in packets of internal priority x and y to map to the same priority x.
Page 166 • PFC is not supported for the N9K-X9408PC-CFP2 line card on Cisco Nexus 9500 Series switches. • Beginning with NX-OS 7.0(3)I1(2), link level flow control and PFC are supported on Cisco Nexus 9300 Series switches and line cards that contain the ALE (Application Leaf Engine).
Page 167: Default Settings For Priority Flow Control
PFC is disabled for interfaces on both sides of back-to-back switches. • Beginning with Cisco NX-OS Release 7.0(3)I7(4), when PFC is received on a lossy priority group (non-configured), the event is recorded in the syslog for subsequent analysis.
Page 168 PFC on the no-drop CoS. Any failures because of a mismatch in the capability of peers causes the PFC not to be enabled. (Cisco NX-OS Release 7.0(3)I3(1) and later) • on—Enables PFC on the local port regardless of the capability of the peers.
Page 169: Enabling Priority Flow Control On A Traffic Class
40 characters. switch(config-cmap-qos)# match { all | any }: Default is match all (if multiple matching statements are present all of them must be matched). Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 170 Policy-map Example: names can contain alphabetic, hyphen, or underscore switch(config)# policy-map type network-qos characters, are case sensitive, and can be up to 40 pfc-qos characters. switch(config-pmap-nqos)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 171 Step 19 interface ethernet slot / number Enters the ethernet interface configuration mode for the selected slot and chassis number. Example: switch(config)# interface ethernet 1/1 switch(config-if)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 172: Configuring A Priority Flow Control Watchdog Interval
Note PFC watchdog is not supported on Cisco Nexus 9500 platform switches with Cisco Nexus 9400, 9500 and 9600 line cards, with the exception of Cisco Nexus 9636PQ line cards (that support the PFC watchdog feature). Note When the PFC watchdog is configured, the following behavior can occur: After the watchdog timer is triggered, the system removes traffic from a non-drop queue and new incoming traffic is not admitted in the ingress buffer.
Page 173 Configuring a Priority Flow Control Watchdog Interval Note For Cisco Nexus 9200 platform switches, Cisco Nexus 9300-EX/FX/FX2 platform switches, and Cisco Nexus 9500 platform switches with -EX or -FX line cards, one of the following calculations is performed to determine...
Page 174 Ingress drops. switch(config)# sh queuing pfc-queue interface ethernet 1/1 detail | QOS GROUP 1 [Active] PFC [YES] PFC-COS [1] +----------------------------------------------------+ Stats +----------------------------------------------------+ Shutdown| Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 175: Configuring Pause Buffer Thresholds And Queue Limit Using Ingress Queuing Policy
For cable lengths that are greater than 100m, the pause buffer size threshold configuration is mandatory and it is required as part of the QoS policy configuration. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 176 Configuring Pause Buffer Thresholds and Queue Limit Using Ingress Queuing Policy Note About queue limits for 100G enabled devices (such as the Cisco Nexus 9300 platform switch with the N9K-M4PC-CFP2 GEM): • The maximum dynamic queue-limit alpha value supported by the device might be greater that 8. However 8 is the maximum alpha value supported.
Page 177: Configuring Qos Pause Buffer And Dynamic Queue Limits
For example, queue-limit percent 1 or queue-limit bytes 100. Configuring QoS Pause Buffer and Dynamic Queue Limits Modifies QoS pause buffers and dynamic queue limits across all the internal interfaces on Cisco Nexus 3164 switches. SUMMARY STEPS 1. configure terminal 2.
Page 178: Verifying The Priority Flow Control Configuration
3 switch(config-pmap-c-qos)# exit switch(config-pmap-qos)# exit switch(config)# class-map type network-qos match-any c1 switch(config-cmap-nqos)# match qos-group 3 switch(config-cmap-nqos)# exit switch(config)# policy-map type network-qos p1 switch(config-pmap-nqos)# class type network-qos c-nq1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 179 Configuring Priority Flow Control Configuration Examples for Priority Flow Control switch(config-pmap-nqos-c)# pause pfc-cos 3 switch(config-pmap-nqos-c)# exit switch(config-pmap-nqos)# exit switch(config)# system qos switch(config-sys-qos)# service-policy type network-qos p1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 180 Configuring Priority Flow Control Configuration Examples for Priority Flow Control Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 181: C H A P T E
1, and z is the internal interface number within the module. Note The number of internal interfaces within a module varies based on the type of the linecard. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 182 Tx Byts | Dropped Pkts | Dropped Byts | Q Depth Byts | +-------------------------------------------------------------------+ QOS GROUP 3 +-------------------------------------------------------------------+ Unicast | OOBFC Unicast Multicast +-------------------------------------------------------------------+ Tx Pkts | Tx Byts | Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 183: Enabling Statistics
You can enable or disable QoS statistics for all interfaces on the device. By default, QoS statistics are enabled. SUMMARY STEPS 1. configure terminal 2. Enable or disable QoS statistics: • Enable QoS statistics: qos statistics • Disable QoS statistics: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 184: Monitoring The Statistics
You can display QoS statistics for all interfaces or a selected interface, data direction, or a QoS type. SUMMARY STEPS 1. show policy-map [policy-map-name] [interface [input | output]] [type {control-plane | network-qos | qos | queuing}] Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 185: Clearing Statistics
(match-any) bandwidth remaining percent 0 Class-map (queuing): c-out-q-default (match-any) bandwidth remaining percent 100 The following example shows how to obtain information about queuing and PFC related counters: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 186 Dropped Pkts | +-------------------------------------------------------------------+ CONTROL QOS GROUP 4 +-------------------------------------------------------------------+ Tx Pkts | Dropped Pkts | +-------------------------------------------------------------------+ SPAN QOS GROUP 5 +-------------------------------------------------------------------+ Tx Pkts | Dropped Pkts | 948| +-------------------------------------------------------------------+ Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 187: C H A P T E
Guidelines and Limitations for Micro-Burst Monitoring The following are the guidelines and limitations for micro-burst monitoring: • Micro-burst monitoring is not supported on the Cisco Nexus 9508 switch (Cisco NX-OS Release NX-OS 7.0(3)F3(3). • Micro-burst monitoring and detection is supported on Cisco Nexus 9300-FX platform switches, Cisco Nexus 9300-FX2 platform switches, and Cisco Nexus 9364C switches.
Page 188 On these switches, micro-burst monitoring is supported on unicast egress queues. It is not supported on multicast, CPU, or span queues. • Beginning with Cisco NX-OS Release 7.0(3)I5(1), micro-burst monitoring is available on the following switches that contain an Application Spine Engine (ASE2, ASE3) or a Leaf Spine Engine (LSE):...
Page 189: Configuring Micro-Burst Detection Per-Queue
Micro-Burst Monitoring Configuring Micro-Burst Detection Per-Queue For more information about EPLD programming to upgrade the FPGA, see the Cisco Nexus 9000 Series FPGA/EPLD Upgrade Release Notes. • The following are guidelines for micro-burst duration on switches that contain a Network Forwarding...
Page 190 Specifies the rise-threshold and the fall-threshold for fall-threshold fall-threshold-bytes bytes micro-burst detection. Example: switch(config-pmap-c-que)# burst-detect rise-threshold 208 bytes fall-threshold 208 bytes Step 5 exit Exits policy-map queue mode. Example: switch(config-pmap-c-que)# exit switch(config-pmap-que)# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 191: Clearing Micro-Burst Detection
Clears micro-burst information from all interfaces or the queue-id]] specified interface. Example: Example • Example for an interface: clear queuing burst-detect interface Eth1/2 • Example for a queue: clear queuing burst-detect interface Eth1/2 queue 7 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 192: Verifying Micro-Burst Detection
Eth 1/2 • Example for a queue: show queuing burst-detect interface Eth 1/2 queue 7 Example of Micro-Burst Detection Output Example output of TOR switch. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 193: Fex Qos Configuration
Verifying the FEX QoS Configuration , on page 198 FEX QoS Configuration Information Note FEX QoS is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3). Note Only 4Q queuing policy model is supported on FEX. When you try to bring up FEX in 8Q queuing policy mode you will get an error message.
Page 194 • Input queuing System Level Hardware Implementation Action Direction: IN Switch Bandwidth Bandwidth Remaining Priority (only level 1) Interface Level Hardware Implementation Action Direction: IN Switch Bandwidth Bandwidth Remaining Priority Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 195: Tcam Carving For Fex Qos
You must free up unused TCAM space to accommodate TCAM carving for FEX QoS. Note FEX QoS is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3). • For FEX QoS TCAM carving for IPv4 traffic, you can use the hardware access-list tcam region fex-qos 256 command.
Page 196: Fex Qos Configuration Example
Service-policy type qos input qos-remark FEX QoS Configuration Example Note FEX QoS is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3). The following are examples of the aspects of a FEX QoS configuration. Classification (system type qos policy) Policies of type qos are applied to classify incoming packets.
Page 197 Match: cos 0 set qos-group 1 Class-map (qos): cos1 (match-all) Match: cos 1 set qos-group 2 Class-map (qos): cos23 (match-all) Match: cos 2-3 set qos-group 3 Class-map (qos): class-default (match-any) Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 198 Flow-control status: rx 0x0, tx 0x0, rx_mask 0x0 qos-group rx pause tx pause masked rx pause -------+-----------+---------+---------+--------------- DSCP to Queue mapping on FEX ----+--+-----+-------+--+--- DSCP to Queue map disabled Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 199 ----- -----SUP 1 start ----- Queuing config per qos_group Interface queueing config valid: no Queueing per qos_group: 00006| |id|bw%|bw_unit|priority grp |00|100|0000000|00000000 grp |01|000|0000000|00000000 grp |02|000|0000000|00000000 grp |03|000|0000000|00000000 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 200 Class-Map ----+----+-----+------+------+--------+--------- 0x03 0x00 0x3c 0x00 0x00 0x00 0x00 0x00 Global FRH: FRH Map: 0x00, Pause Class Map: 0x00 Xoff Threshold: 0, Total Credits: 0 Pause configuration: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 201 Rx Error (WR_RCV_ERR): OQ statistics: Packets flushed: Packets timed out: 0 Pause statistics: Rx PFC Xoff Tx PFC Xoff ------+-----------------------+---------------------- Rx Xoff: Rx Xon: Tx Xoff: Tx Xon: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 202 Description: Classifier for Ingress queue 1 match qos-group 1 class-map type queuing match-any c-in-q-default Description: Classifier for Ingress default queue match qos-group 0 switch# Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 203 Media type 3 Port speed 10000 MAC addr 00:00:00:00:00:00 Port state: , Down fabric_num 0, ctrl_vntag 0 ctrl_vlan 0, vntag_etype 0 Untagged COS config valid: no Untagged COS dump: Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 204 TX| 0x00|000000003|0000000 class |05|0x20|005|000|0000000|0000007|0000| TX| 0x00|000000003|0000000 class |06|0x40|006|000|0000000|0000007|0000| TX| 0x00|000000003|0000000 class |07|0x80|007|000|0000000|0000007|0000| TX| 0x00|000000003|0000000 -----SUP 1 end ----- PFC 1 (enabled), net_port 0x0 END of PI SECTION NIF0/0/1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 205 0x00 0xff 0x00 0xff OQ configuration: Credit Quanta: 1, IPG Adjustment: 0 PQ0 En: 0, PQ0 Class: 0 PQ1 En: 0, PQ1 Class: 0 Class XoffToMap CrDec ------+----------+---+---+---+----+----+----+-----+---- Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 206 Output queuing (system type queuing output policy) Note System Output queuing is applied on HIF Ports for NIF to HIF traffic. • Policy map (system defined policy map): switch# show policy-map type queuing default-out-policy Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 207 (match-any) bandwidth percent 20 Class-map (queuing): c-out-q-default (match-any) bandwidth percent 10 switch# show queuing interface ethernet 101/1/1 slot ======= Ethernet101/1/1 queuing information: Input buffer allocation: Qos-group: ctrl Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 208 DSCP to Queue mapping on FEX ----+--+-----+-------+--+--- DSCP to Queue map disabled FEX TCAM programmed successfully switch# switch# attach fex 101 fex-101# show platform software qosctrl port 0 0 hif 1 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 209 |04|000|0000000|00000000 grp |05|000|0000000|00000000 Scheduling Classes 00008| |id|cbmp|qid|bw%|nor_bw%|bw_unit|prio|dir |q2cos|class_grp|wk_gmap class |00|0x01|000|000|0000000|0000007|0001| TX| 0x80|000000000|0000000 class |01|0x02|001|000|0000000|0000007|0001| TX| 0x00|000000000|0000000 class |02|0x04|002|000|0000000|0000007|0000| TX| 0x08|000000002|0000000 class |03|0x08|003|100|0000100|0000007|0000| TX| 0xf7|000000003|0000000 class |04|0x10|004|000|0000000|0000007|0000| TX| 0x00|000000003|0000000 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 210 Pause configuration: PFC disabled Rx PFC CoS map: 0x00, Tx PFC CoS map: 0x00 Index CoS-to-Class Class-to-CoS------+-------------+------------ 0x00 0xff 0x00 0xff 0x00 0xff 0x00 0xff 0x00 0xff 0x00 0xff Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 211 Tx Xoff: Tx Xon: Rx PFC: Tx PFC: Rx Xoff Status: 0x00 Tx Xoff Status: 0x00 RdPort Class Head Tail QCount RealQCountRx ---+-------+------+------+------+-------+------------ 3113 9348 11057 4864 5356 4257 Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 212: Verifying The Fex Qos Configuration
[qos | queuing] Displays information about all configured policy maps of type qos or queuing on the system. show queuing interface ethernet Displays information about queuing on the ethernet interface. Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 213: Appendix B Additional References
A P P E N D I X Additional References This appendix contains additional information related to implementing QoS on the Cisco NX-OS device. This appendix includes the following sections: • RFCs, on page 199 RFCs RFCs Title RFC 2474...
Page 214 Additional References Additional References Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x...
Page 215 52, 53 176, 177 interface ethernet service-policy type queuing output 176, 177 show hardware access-list tcam template Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x IN-1...
Page 216 INDEX Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide, Release 7.x IN-2...

This manual is also suitable for:

N9k-c9272q N9k-c9332pq N9k-c93120tx N9k-x9464px N9k-x9432pq

Cisco Nexus 9000 Series Configuration Manual

Appendix B Additional References

Quick Links

Related Manuals for Cisco Nexus 9000 Series

Summary of Contents for Cisco Nexus 9000 Series

This manual is also suitable for:

Table of Contents