Cisco Nexus 9000 Series Configuration Manual
  1. Manuals
  2. Brands
  3. Cisco Manuals
  4. Network Router
  5. Nexus 9000 Series
  6. Configuration manual

Cisco Nexus 9000 Series Configuration Manual

Nx-os vxlan
Hide thumbs Also See for Nexus 9000 Series:
Table of Contents

Advertisement

Quick Links

See also: Troubleshooting Manual , Configuration Manual
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release
7.x
First Published: 2015-01-27
Last Modified: 2017-02-17
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883

Advertisement

Table of Contents
loading

Related Manuals for Cisco Nexus 9000 Series

Summary of Contents for Cisco Nexus 9000 Series

  • Page 1 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release First Published: 2015-01-27 Last Modified: 2017-02-17 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883...
  • Page 2 This product includes software written by Tim Hudson (tjh@cryptsoft.com). https:/ Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: /www.cisco.com/go/trademarks .

  • Page 3: Table Of Contents

    C H A P T E R 3 Information About VXLAN Guidelines and Limitations for VXLAN Considerations for VXLAN Deployment VPC Considerations for VXLAN Deployment Network Considerations for VXLAN Deployments Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 4 Notes for EVPN Convergence Considerations for VXLAN BGP EVPN Deployment VPC Considerations for VXLAN BGP EVPN Deployment Network Considerations for VXLAN Deployments Considerations for the Transport Network BGP EVPN Considerations for VXLAN Deployment Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 5 Configuring NGOAM Profile NGOAM Authentication Configuring VXLAN EVPN Multihoming C H A P T E R 6 VXLAN EVPN Multihoming Overview Introduction to Multihoming BGP EVPN Multihoming BGP EVPN Multihoming Terminology Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 6 DHCP Relay in VXLAN BGP EVPN A P P E N D I X B DHCP Relay in VXLAN BGP EVPN Overview DHCP Relay in VXLAN BGP EVPN Example Basic VXLAN BGP EVPN Configuration Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 7 IPv6 Across a VXLAN EVPN Fabric A P P E N D I X D Overview of IPv6 Across a VXLAN EVPN Fabric Configuring IPv6 Across a VXLAN EVPN Fabric Example Show Command Examples Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 8 Contents Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x viii...

  • Page 9: Document Conventions

    • Documentation Feedback, page x • Obtaining Documentation and Submitting a Service Request, page xi Audience This publication is for network administrators who install, configure, and maintain Cisco Nexus switches. Document Conventions Command descriptions use the following conventions: Convention Description...

  • Page 10: Documentation Feedback

    An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line. Related Documentation for Cisco Nexus 9000 Series Switches The entire Cisco Nexus 9000 Series switch documentation set is available at the following URL: http://www.cisco.com/en/US/products/ps13386/tsd_products_support_series_home.html Documentation Feedback To provide technical feedback on this document, or to report an error or omission, please send your comments to nexus9k-docfeedback@cisco.com.

  • Page 11: Obtaining Documentation And Submitting A Service Request

    Subscribe to What's New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.
  • Page 12 Preface Obtaining Documentation and Submitting a Service Request Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 13: Chapter

    • New and Changed Information, page 1 New and Changed Information This table summarizes the new and changed features for the Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide and where they are documented. Table 1: New and Changed Features...
  • Page 14 Added support for displaying 7.0(3)I2(2) Verifying the VXLAN tracking route information. Configuration LACP tunneling support for Added support for VXLAN 7.0(3)I2(2) Configuring Q-in-VNI with VXLAN with LACP tunneling. LACP Tunneling Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 15 Static MAC for VXLAN VTEP Enables the configuration of 7.0(3)I1(2) Configuring Static MAC for support static MAC addresses behind a VXLAN VTEP peer VTEP on Cisco Nexus 9300 Series switches. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 16 VXLAN BGP EVPN support Enables the learning of remote 7.0(3)I1(1) Configuring VXLAN BGP VTEPs, overlay MACs, and EVPN routes through the BGP EVPN control plane protocol on Cisco Nexus 9300 Series switches. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 17: Chapter

    Layer 2 segments from the shared transport network. When deployed as a VXLAN gateway, Cisco Nexus 9000 switches can connect VXLAN and classic VLAN segments to create a common forwarding domain so that tenant devices can reside in both environments.

  • Page 18: Vxlan Encapsulation And Packet Format

    It is not uncommon that virtual machines in a VXLAN segment need to access services provided by devices in a classic VLAN segment. This type of VXLAN-to-VLAN connectivity is enabled by using a VXLAN gateway. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 19: Vpc Consistency Check For Vpc Vteps

    A VXLAN gateway is a VTEP device that combines a VXLAN segment and a classic VLAN segment into one common Layer 2 domain. A Cisco Nexus 9000 Series Switch can function as a hardware-based VXLAN gateway. It seamlessly connects VXLAN and VLAN segments as one forwarding domain across the Layer 3 boundary without sacrificing forwarding performance.

  • Page 20: Static Ingress Replication

    • Remote peers are statically configured. • Multi-destination packets are unicast encapsulated and delivered to each of the statically configured remote peers. Cisco NX-OS supports multiple remote peers in one segment and also allows the same remote peer in Note multiple segments.

  • Page 21: Bud Node Topology

    The Cisco Nexus 9000 Series switches provide support for the bud node topology. The application leaf engine (ALE) of the device enables it to be a VXLAN VTEP device and an IP transit device at the same time so the device can become a bud node.
  • Page 22 The distributed anycast gateway functionality will be used to facilitate flexible workload placement, and optimal traffic across the L3 core network. The overlay network that will be used is based on VXLAN. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 23: Chapter

    Information About VXLAN Guidelines and Limitations for VXLAN VXLAN has the following guidelines and limitations: • Beginning with Cisco NX-OS Release 7.0(3)I6(1), a new CLI command lacp vpc-convergence is added for better convergence of Layer 2 EVPN VXLAN: interface port-channel10...
  • Page 24 IGMP snooping on VXLAN enabled VLANs is not supported in Cisco Nexus 3232C and 3264Q switches. VXLAN with flood and learn and Layer 2 EVPN is supported in Cisco Nexus 3232C and 3264Q switches. • Bind NVE to a loopback address that is separate from other loopback addresses that are required by Layer 3 protocols.
  • Page 25 Configuring VXLAN Guidelines and Limitations for VXLAN • The VXLAN UDP port number is used for VXLAN encapsulation. For Cisco Nexus NX-OS, the UDP port number is 4789. It complies with IETF standards and is not configurable. • For 7.0(3)I2(1) and later, VXLAN is supported on Cisco Nexus 9500 Series switches with the following linecards: ◦...

  • Page 26: Considerations For Vxlan Deployment

    Cisco Nexus 9200 switches, Cisco Nexus 9300-EX switches, or Cisco Nexus 9500 switches with N9K-X9732C-EX line cards. • When configuring BGP-EVPN on Cisco Nexus 9300-EX switches and Cisco Nexus 9500 switches with N9K-X9732C-EX line cards, use the system routing template-vxlan-scale command. Performing this...

  • Page 27: Vpc Considerations For Vxlan Deployment

    Configuring VXLAN Guidelines and Limitations for VXLAN step requires a reload of the switch. This command is not applicable on Cisco Nexus 9200 switches, Cisco Nexus 9300 switches, and Cisco Nexus 9500 switches with N9K-X9564PX, N9K-X9564TX, and N9K-X9536PQ line cards.
  • Page 28 • The VPC peer-gateway feature must be enabled on both peers. As a best practice, use peer-switch, peer gateway, ip arp sync, ipv6 nd sync configurations for improved convergence in VPC topologies. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 29 In BUD node topologies, the backup SVI needs to be added as a static OIF for each Note underlay multicast group. The SVI must be configured on both VPC peers and requires PIM to be enabled. Note Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 30: Network Considerations For Vxlan Deployments

    • Using the ip forward command enables the VTEP to forward the VXLAN de-capsulated packet destined to its router IP to the SUP/CPU. • Before configuring it as an SVI, the backup VLAN needs to be configured on Cisco Nexus 9200 Series switches as an infra-VLAN with the system nve infra-vlans command.

  • Page 31: Considerations For The Transport Network

    • Throughout the transport network: ◦ Enable and configure IP multicast.* • When using SVI uplinks with VXLAN enabled on Cisco Nexus 9200 Series switches and Cisco Nexus 9300-EX switches, use the system nve infra-vlans command to specify the VLANs that are used for uplink SVI.

  • Page 32: Mapping Vlan To Vxlan Vni

    Refer to the VLAN counters on the translated VLAN and not on the ingress (incoming) VLAN. • Port VLAN mapping is supported on Cisco Nexus 9300 Series switches. Beginning with Cisco NX-OS Release 7.0(3)I6(1), port VLAN mapping is supported on Cisco Nexus 9300-EX Series switches.
  • Page 33 • Port VLAN mapping is not supported on Cisco Nexus 9200 Series switches. Beginning with Cisco NX-OS Release 7.0(3)I6(1), port VLAN switching is supported on Cisco Nexus 9500 and 9300 platform switches. However, PV routing is not supported on Cisco Nexus 9500 and 9300 platform switches.
  • Page 34 VLANs. Step 5 [no] switchport vlan Removes all VLAN mappings configured on the interface. mapping all Step 6 copy running-config (Optional) Copies the running configuration to the startup configuration. startup-config Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 35: Configuring Inner Vlan And Outer Vlan Mapping On A Trunk Port

    • Inner and outer VLAN cannot be on the trunk allowed list on a port where inner VLAN and outer VLAN is configured. For example: switchport vlan mapping 11 inner 12 111 switchport trunk allowed vlan 11-12,111 /***Not valid because 11 is outer VLAN and 12 is inner VLAN.***/ Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 36 11 inner 12 111 switch(config-if)# switchport trunk allowed vlan 101-170 switch(config-if)# no shutdown switch(config-if)# show mac address-table dynamic vlan 111 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 37: Creating And Configuring An Nve Interface And Associate Vnis

    Configuring Static MAC for VXLAN VTEP Static MAC for VXLAN VTEP is supported on Cisco Nexus 9300 Series switches with flood and learn. This feature enables the configuration of static MAC addresses behind a peer VTEP. Static MAC cannot be configured for a control plane with a BGP EVPN-enabled VNI.

  • Page 38: Disabling Vxlans

    Disables the VXLAN feature. Step 4 copy running-config startup-config (Optional) Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 39: Configuring Bgp Evpn Ingress Replication

    Only 1 NVE interface is allowed on the Note switch. Step 3 member vni [vni-id | vni-range] Maps VXLAN VNIs to the NVE interface. Step 4 ingress-replication protocol static Enables static ingress replication for the VNI. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 40: Configuring Q-In-Vni

    Notes about configuring a Q-in-VNI: • Q-in-VNI only supports VXLAN bridging. It does not support VXLAN routing. • The dot1q mode does not support 40G ports on Cisco Nexus 9300 Series and Cisco Nexus 9500 Series switches. • Q-in-VNI does not support FEX.
  • Page 41 • The following is an example of configuring a Q-in-VNI (NX-OS 7.0(3)I3(1) and later releases): switch# config terminal switch(config)# interface ethernet 1/4 switch(config-if)# switchport mode dot1q-tunnel switch(config-if)# switchport access vlan 10 switch(config-if)# spanning-tree bpdufilter enable switch(config-if)# Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 42: Configuring Selective Q-In-Vni

    VLAN’s SVI that is configured on the selective Q-in-VNI port (no VXLAN). Beginning with Cisco NX-OS Release 7.0(3)I5(2), selective Q-in-VNI is supported on both vPC and non-vPC ports on Cisco Nexus 9300-EX Series switches. This feature is not supported on Cisco Nexus 9300 Series and 9200 Series switches.
  • Page 43 10050 mcast-group 230.1.1.1 • See the following example for the native VLAN configuration: vlan 150 interface vlan150 no shutdown ip address 150.1.150.6/24 ip pim sparse-mode Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 44: Configuring Q-In-Vni With Lacp Tunneling

    • The following is an example of configuring a Q-in-VNI for LACP tunneling (NX-OS 7.0(3)I2(2) and earlier releases): switch# config terminal switch(config)# interface ethernet 1/4 switch(config-if)# switchport mode dot1q-tunnel switch(config-if)# switchport access vlan 10 switch(config-if)# spanning-tree bpdufilter enable switch(config-if)# interface nve1 switch(config-if)# overlay-encapsulation vxlan-with-tag tunnel-control-frames Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 45 • No MAC address-table notification for mac-move. • As a best practice, configure a fast LACP rate on the interface where the LACP port is configured. Otherwise the convergence time is approximately 90 seconds. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 46 ◦ To avoid saturating the MAC, you should turn off/disable learning of VLANS. • Configuring Q-in-VNI to tunnel LACP packets is not supported for VXLAN EVPN. • The number of port-channel members supported is the number of ports supported by the VTEP. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 47: Configuring Fhrp Over Vxlan

    Starting with Release 7.0(3)I5(1), you can configure First Hop Redundancy Protocol (FHRP) over VXLAN on Cisco Nexus 9000 Series switches. The FHRP provides a redundant Layer 3 traffic path. It provides fast failure detection and transparent switching of the traffic flow. The FHRP avoids the use of the routing protocols on all the devices.

  • Page 48: Only Supported Deployments For Fhrp Over Vxlan

    See the following illustrations for only supported deployments for FHRP over VXLAN protocols. Figure 3: FHRP over VXLAN Leafs as Layer 3 Gateway Figure 4: FHRP over VXLAN Spine as Layer 3 Gateway Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 49 FHRP operates in active/active. The VNI mapped to the VLAN must be configured on the NVE interface and it is associated with the used BUM replication mode (Multicast or Ingress Replication). Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 50: New Supported Topology For Configuring Fhrp Over Vxlan

    Configuring VXLAN Configuring FHRP Over VXLAN New Supported Topology for Configuring FHRP Over VXLAN Configuring FHRP over VXLAN is supported on the following Cisco Nexus 9000 Series switches and line cards: • Cisco Nexus 9300 Series switches • N9K-X9536PQ line cards •...

  • Page 51: Configuring Igmp Snooping Over Vxlan

    • Beginning with Cisco NX-OS Release 7.0(3)I5(2), VXLAN IGMP snooping is supported on Cisco Nexus 9300 Series switches and Cisco Nexus 9500 Series switches with N9K-X9732C-EX line cards. • By default, unknown multicast traffic gets flooded to the VLAN domains on Cisco Nexus 9300 Series switches.

  • Page 52: Configuring Igmp Snooping Over Vxlan

    For VXLAN IGMP snooping functionality, the ARP-ETHER TCAM must be configured in the double-wide mode using the hardware access-list tcam region arp-ether 256 double wide command for Cisco Nexus 9300 switches. This command is not required for Cisco Nexus 9300-EX switches..
  • Page 53 Displays logging level. show tech-support nve Displays related NVE tech-support information. show run interface nve x Displays NVE overlay interface configuration. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 54 VXLAN VLAN logical port VP count is 10*10 = 100. Table 4: Display VXLAN configuration information (Release 7.0(3)I2(2) and later) Command Purpose Displays tracking information for running-config. show run track Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 55: Example Of Vxlan Bridging Configuration

    20.1.1.1/30 switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0 switch-vtep-1(config-if)# ip pim sparse-mode switch-vtep-1(config)# feature nv overlay switch-vtep-1(config)# feature vn-segment-vlan-based switch-vtep-1(config)# interface e1/1 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 56 10 switch-vtep-2(config-if)# no shutdown switch-vtep-2(config)# interface nve1 switch-vtep-2(config-if)# no shutdown switch-vtep-2(config-if)# source-interface loopback0 switch-vtep-2(config-if)# member vni 10000 mcast-group 230.1.1.1 switch-vtep-2(config)# vlan 10 switch-vtep-2(config-vlan)# vn-segment 10000 switch-vtep-2(config-vlan)# exit Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 57 200.200.9.9 switch-vtep-1(config-vlan)# exit switch-vtep-1# show nve vni ingress-replication Interface VNI show nve vni ingress-replication Interface VNI Replication List Up Time --------- -------- ----------------- ------- Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 58 Replication List Up Time --------- -------- ----------------- ------- nve1 10011 200.200.8.8 07:42:23 200.200.10.10 07:42:23 nve1 10012 200.200.8.8 07:42:23 • For a vPC VTEP configuration, the loopback address requires a secondary IP. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 59 10 mode active switch-vtep-1(config-if)# no shutdown switch-vtep-1(config)# interface nve1 switch-vtep-1(config-if)# no shutdown switch-vtep-1(config-if)# source-interface loopback0 switch-vtep-1(config-if)# member vni 10000 mcast-group 230.1.1.1 switch-vtep-1(config)# vlan 10 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 60 10 switch-vtep-3(config-if)# no shutdown switch-vtep-3(config)# interface nve1 switch-vtep-3(config-if)# no shutdown switch-vtep-3(config-if)# source-interface loopback0 switch-vtep-3(config-if)# member vni 10000 mcast-group 230.1.1.1 switch-vtep-3(config)# vlan 10 switch-vtep-3(config-vlan)# vn-segment 10000 switch-vtep-3(config-vlan)# exit Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 61 Example of VXLAN Bridging Configuration The secondary IP is used by the emulated VTEP for VXLAN. Note Ensure that all configurations are identical between the VPC primary and VPC secondary. Note Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 62 Configuring VXLAN Example of VXLAN Bridging Configuration Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 63: Configuring Vxlan Bgp Evpn

    Guidelines and Limitations for VXLAN BGP EVPN VXLAN BGP EVPN has the following guidelines and limitations: • Beginning with Cisco NX-OS Release 7.0(3)I6(1), you can configure EVPN over segment routing or MPLS. See the Cisco Nexus 9000 Series NX-OS Label Switching Configuration Guide, Release 7.x more information.
  • Page 64 • VXLAN BGP EVPN does not support an NVE interface in a non-default VRF. • It is recommended to configure a single BGP session over the loopback for an overlay BGP session. • For Cisco Nexus 9500 Series switches (7.0(3)I2(1) and later), VXLAN BGP EVPN is available only in the default routing mode.

  • Page 65: Notes For Evpn Convergence

    For VXLAN routing support, a 40G uplink module is required. Note • The VXLAN UDP port number is used for VXLAN encapsulation. For Cisco Nexus NX-OS, the UDP port number is 4789. It complies with IETF standards and is not configurable.

  • Page 66: Considerations For Vxlan Bgp Evpn Deployment

    • Every tenant VRF needs a VRF overlay VLAN and SVI for VXLAN routing. • For Cisco Nexus 9500 Series switches (7.0(3)I1(2) and later), always reserve the VRF overlay VLANs for NVE at the global level using the system vlan nve-overlay id range command. This example shows...

  • Page 67: Vpc Considerations For Vxlan Bgp Evpn Deployment

    Configuring VXLAN BGP EVPN Guidelines and Limitations for VXLAN BGP EVPN Beginning with Cisco NX-OS Release 7.0(3)I5(2), this command applies to Cisco Nexus Note 9200 and 9300-EX Series switches. NVE overlay VLANs should be used for VXLAN Layer 3 routing only. They should not be used for regular VLANs.
  • Page 68 SVI is required to be enabled across peer-link and also configured with PIM. This provides a backup routing path in the case when VTEP loses complete connectivity to the spine. Remote peer reachability is re-routed over the peer-link in this case. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 69: Network Considerations For Vxlan Deployments

    • ECMP and LACP Hashing Algorithms in the Transport Network As described in a previous section, Cisco Nexus 9000 Series Switches introduce a level of entropy in the source UDP port for ECMP and LACP hashing in the transport network. As a way to augment this...

  • Page 70: Considerations For The Transport Network

    • Throughout the transport network: ◦ Enable and configure IP multicast.* • When using SVI uplinks with VXLAN enabled on Cisco Nexus 9200 Series switches and Cisco Nexus 9300-EX switches, use the system nve infra-vlans command to specify the VLANs that are used for uplink SVI.

  • Page 71: Bgp Evpn Considerations For Vxlan Deployment

    Cisco Nexus 9500 Series switches. Beginning with Cisco NX-OS Release 7.0(3)I5(2), this command applies to Cisco Nexus 9200 and 9300-EX Series switches. NVE overlay VLANs should be used for Note VXLAN Layer 3 routing only. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 72: Configuring Vxlan Bgp Evpn

    Command or Action Purpose Step 1 Enable VLAN-based VXLAN feature vn-segment Step 2 Enable VXLAN feature nv overlay Step 3 Enable the EVPN control plane for VXLAN. nv overlay evpn Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 73: Configuring Vlan And Vxlan Vni

    Configure address family for IPv6. address-family ipv6 unicast Step 8 route-target both auto Note Specifying the auto option is applicable only for IBGP. Manually configured route targets are required for EBGP. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 74: Configuring Svi For Hosts For Vxlan Routing

    Configures a Layer 3 VNI under a VRF overlay VLAN. (A VRF overlay VLAN is a VLAN that is not associated with any server facing ports. All VXLAN VNIs that are mapped to a VRF, need to have their own internal VLANs allocated to it.) Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 75: Configuring Anycast Gateway For Vxlan Routing

    Required for VXLAN routing Note only. Step 4 member vni vni Add Layer 2 VNIs to the tunnel interface. Step 5 mcast-group address Configure the mcast group on a per-VNI basis Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 76: Configuring Bgp On The Vtep

    Configuring RD and Route Targets for VXLAN Bridging Procedure Command or Action Purpose Step 1 evpn Configure VRF. Step 2 vni number l2 Only Layer 2 VNIs need to be Note specified. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 77: Configuring Vxlan Evpn Ingress Replication

    Add Layer-3 VNIs, one per tenant VRF, to the overlay. Required for VXLAN routing Note only. Step 4 member vni vni Add Layer 2 VNIs to the tunnel interface. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 78: Configuring Bgp For Evpn On The Spine

    Required for eBGP. Allows the spine to retain and Note advertise all EVPN routes when there are no local VNI configured with matching import route targets. Step 6 neighbor address remote-as Define neighbor. number Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 79: Suppressing Arp

    Step 2 interface nve 1 Create the network virtualization endpoint (NVE) interface. Step 3 member vni vni-id Specify VNI ID. Step 4 suppress-arp Configure to suppress ARP under Layer 2 VNI. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 80: Disabling Vxlans

    Duplicate Detection for IP and MAC Addresses Cisco NX-OS supports duplicate detection for IP and MAC addresses. This enables the detection of duplicate IP or MAC addresses based on the number of moves in a given time-interval (seconds).
  • Page 81 The range is 2 to 36000 seconds; default is 180 seconds. Detects duplicate host addresses (limited switch(config)# l2rib dup-host-mac-detection 100 10 to 100 moves) in a period of 10 seconds. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 82: Verifying The Vxlan Bgp Evpn Configuration

    Note Although the show ip bgp command is available for verifying a BGP configuration, as a best practice, it is preferable to use the show bgp command instead. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 83: Example Of Vxlan Bgp Evpn (Ebgp)

    Figure 9: VXLAN BGP EVPN Topology (EBGP) EBGP between Spine and Leaf • Spine (9504-A) ◦ Enable the EVPN control plane nv overlay evpn ◦ Enable the relevant protocols feature bgp feature pim Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 84 40.1.1.1 remote-as 200 update-source loopback0 ebgp-multihop 3 address-family l2vpn evpn disable-peer-as-check send-community extended route-map permitall out ◦ Configure the BGP underlay. neighbor 192.168.1.43 remote-as 200 address-family ipv4 unicast allowas-in Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 85 100 router-id 20.1.1.1 address-family l2vpn evpn retain route-target all neighbor 30.1.1.1 remote-as 200 update-source loopback0 ebgp-multihop 3 address-family l2vpn evpn disable-peer-as-check send-community extended route-map permitall out neighbor 40.1.1.1 remote-as 200 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 86 50.1.1.1/32 ip pim sparse-mode ◦ Configure interfaces for Spine-leaf interconnect interface Ethernet2/2 no switchport load-interval counter 1 5 ip address 192.168.1.22/24 ip pim sparse-mode no shutdown Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 87 4.2.2.1/24 ipv6 address 4:2:0:1::1/64 fabric forwarding mode anycast-gateway ◦ Configure ACL TCAM region for ARP suppression hardware access-list tcam region arp-ether 256 double-wide Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 88 ◦ Enable the EVPN control plane functionality and the relevant protocols feature telnet feature nxapi feature bash-shell feature scp-server nv overlay evpn feature bgp feature pim feature interface-vlan feature vn-segment-vlan-based Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 89 4.1.1.1/24 ipv6 address 4:1:0:1::1/64 fabric forwarding mode anycast-gateway interface Vlan1002 no shutdown vrf member vxlan-900001 ip address 4.2.2.1/24 ipv6 address 4:2:0:1::1/64 fabric forwarding mode anycast-gateway Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 90 200 router-id 40.1.1.1 neighbor 10.1.1.1 remote-as 100 update-source loopback0 ebgp-multihop 3 allowas-in send-community extended address-family l2vpn evpn allowas-in send-community extended neighbor 20.1.1.1 remote-as 100 update-source loopback0 ebgp-multihop 3 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 91 2001001 l2 rd auto route-target import auto route-target export auto vni 2001002 l2 rd auto route-target import auto route-target export auto Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 92: Example Of Vxlan Bgp Evpn (Ibgp)

    Figure 10: VXLAN BGP EVPN Topology (IBGP) IBGP between Spine and Leaf • Spine (9504-A) ◦ Enable the EVPN control plane nv overlay evpn ◦ Enable the relevant protocols feature ospf feature bgp feature pim Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 93 • Spine (9504-B) ◦ Enable the EVPN control plane and the relevant protocols feature telnet feature nxapi feature bash-shell feature scp-server nv overlay evpn feature ospf Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 94 40.1.1.1 remote-as 65535 update-source loopback0 address-family l2vpn evpn send-community both route-reflector-client • Leaf (9396-A) ◦ Enable the EVPN control plane nv overlay evpn Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 95 ◦ Configure VRF overlay VLAN/SVI for the VRF interface Vlan101 no shutdown vrf member vxlan-900001 ◦ Create VLAN and provide mapping to VXLAN vlan 1001 vn-segment 2001001 vlan 1002 vn-segment 2001002 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 96 ◦ Configure BGP router bgp 65535 router-id 30.1.1.1 neighbor 10.1.1.1 remote-as 65535 update-source loopback0 address-family l2vpn evpn send-community both neighbor 20.1.1.1 remote-as 65535 update-source loopback0 address-family l2vpn evpn send-community both vrf vxlan-900001 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 97 ◦ Create VRF and configure VNI vrf context vxlan-900001 vni 900001 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn address-family ipv6 unicast route-target both auto route-target both auto evpn Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 98 192.168.4.22/24 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown interface Ethernet2/3 no switchport ip address 192.168.2.23/24 ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 99: Example Show Commands

    --------- -------- ----------------- ----- ---- ------------------ ----- nve1 900001 L3 [vxlan-900001] nve1 2001001 225.4.0.1 L2 [1001] nve1 2001002 225.4.0.1 L2 [1002] • show ip arp suppression-cache detail 9396-B# show ip arp suppression-cache detail Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 100 Topology Mac Address Prod Next Hop (s) ----------- -------------- ------ --------------- 0000.8816.b645 BGP 40.0.0.2 0001.0000.0033 Local Ifindex 4362086 0001.0000.0035 Local Ifindex 4362086 0011.0000.0034 BGP 40.0.0.2 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 101 • show l2route evpn mac-ip all leaf3# show l2route evpn mac-ip all Topology ID Mac Address Prod Host IP Next Hop (s) ----------- -------------- ---- ------------------------------------------------------ 0011.0000.0034 BGP 5.1.3.2 40.0.0.2 0011.0000.0034 BGP 5.1.3.2 40.0.0.2 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 102 Configuring VXLAN BGP EVPN Example Show Commands Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 103: Configuring Vxlan Oam

    OAM EtherType or by using a well-known reserved source MAC address in the OAM packets depending on the implementation on different platforms. This constitutes a signature for recognition of the VXLAN OAM packets. The VXLAN OAM tools are categorized as shown in table below. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 104: Loopback (Ping) Message

    VM. The ping message supports the following reachability options: Ping Check the network reachability (Ping command): • From Leaf 1 (VTEP 1) to Leaf 2 (VTEP 2) (ICMP or NVO3 draft Tissa channel) Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 105: Traceroute Or Pathtrace Message

    ECMP paths between a source and destination switch. The TTL expiry message may also be generated by the intermediate switches for the actual data frames. The same payload of the original path trace request is preserved for the payload of the response. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 106 (for example, ingress interface and egress interface). These packets terminate at VTEP and they does not reach the host. Therefore, only the VTEP responds. Figure 12: Traceroute Message Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 107: Configuring Vxlan Oam

    Forwarding Engine (NFE), complete this verification step. After entering the command, perform a lookup for entry/eid with data=0x8902 under EtherType. Step 5 # show system internal access-list (Optional) tcam ingress start-idx <hardware index> count 1 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 108 The source ip-address 1.1.1.1 used in the above example is a loopback interface that is configured on Leaf Note 1 in the same VRF as the destination ip-address. For example, the VRF in this example is vni-31000. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 109 Path trace Request to peer ip 209.165.201.4 source ip 209.165.201.2 Sender handle: 46 TTL Code Reply IngressI/f EgressI/f State ====================================================================== 1 !Reply from 209.165.201.3, Eth5/5/1 Eth5/5/2 UP/UP 2 !Reply from 209.165.201.4, Eth1/3 Unknown UP/DOWN Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 110: Configuring Ngoam Profile

    See the following examples for configuring an NGOAM profile and for configuring NGOAM flow. switch(config)# ngoam profile 1 oam-channel 1 flow forward payload pad 0x2 sport 12345, 54321 switch(config-ngoam-profile)#flow {forward } Enters config-ngoam-profile-flow submode to configure forward flow entropy specific information Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 111: Ngoam Authentication

    Configuring VXLAN OAM NGOAM Authentication NGOAM Authentication NGOAM provides the interface statistics in the pathtrace response. Beginning with Cisco NX-OS Release 7.0(3)I6(1), NGOAM authenticates the pathtrace requests to provide the statistics by using the HMAC MD5 authentication mechanism. NGOAM authentication validates the pathtrace requests before providing the interface statistics. NGOAM authentication takes effect only for the pathtrace requests with req-stats option.
  • Page 112 Input Stats: PktRate:0 ByteRate:0 Load:0 Bytes:339580108 unicast:14658 mcast:307587 bcast:67 discards:0 errors:3 unknown:0 bandwidth:42949672970000000 Output Stats: PktRate:0 ByteRate:0 load:0 bytes:237405790 unicast:2929 mcast:535716 bcast:10408 discards:0 errors:0 bandwidth:42949672970000000 2 !Reply from 12.0.22.1, Eth1/17 Unknown UP / DOWN Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 113: Configuring Vxlan Evpn Multihoming

    Cisco Nexus platforms support vPC-based multihoming, where a pair of switches act as a single device for redundancy and both switches function in an active mode. With Cisco Nexus 9000 Series switches in VXLAN BGP EVPN environment, there are two solutions to support Layer 2 multihoming; the solutions are based on the Traditional vPC (emulated or virtual IP address) and the BGP EVPN techniques.

  • Page 114: Bgp Evpn Multihoming Terminology

    VTEP withdraws the corresponding set of Ethernet Auto-Discovery per ES routes. Ethernet Segment Route is the other route type that is being used by Cisco NX-OS software with EVPN multihoming, mainly for Designated Forwarder (DF) election for the BUM traffic. If the Ethernet Segment is multihomed, the presence of multiple DFs could result in forwarding the loops in addition to the potential packet duplication.

  • Page 115: Evpn Multihoming Redundancy Group

    ESI configured MAC address value to the access switch. LACP is not mandated along with ESI. A given ESI interface (PO) shares the same ESI ID across the VTEPs in the group. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 116: Guidelines And Limitations For Vxlan Evpn Multihoming

    ID under the same PO, LACP brings down one of the links (first link that comes online stays up). By default, on most Cisco Nexus platforms, LACP sets a port to the suspended state if it does not receive an LACP PDU from the peer.

  • Page 117: Vxlan Evpn Multihoming Configuration Examples

    BUM traffic duplication on the shared ES POs. Example: hardware access-list tcam region vpc-convergence 256 VXLAN EVPN Multihoming Configuration Examples See the sample VXLAN EVPN multihoming configuration on the switches: Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 118 9216 ip address 10.1.1.6/30 ip pim sparse-mode no shutdown interface port-channel11 switchport mode trunk switchport access vlan 1001 switchport trunk allowed vlan 901-902,1001-1050 ethernet-segment 2011 system-mac 0000.0000.2011 mtu 9216 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 119: Configuring Layer 2 Gateway Stp

    Configuring Layer 2 Gateway STP Layer 2 Gateway STP Overview Beginning with Cisco NX-OS Release 7.0(3)I5(2), EVPN multihoming is supported with the Layer 2 Gateway Spanning Tree Protocol (L2G-STP). The Layer 2 Gateway Spanning Tree Protocol (L2G-STP) builds a loop-free tree topology. However, the Spanning Tree Protocol root must always be in the VXLAN fabric. A bridge ID for the Spanning Tree Protocol consists of a MAC address and the bridge priority.

  • Page 120: Enabling Layer 2 Gateway Stp On A Switch

    VTEPs. Performing this action results in conflicts and disputes because the non-Layer 2 Gateway STP VTEP keeps sending BPDUs and it can steer the root outside. • Keep the current edge and the BPDU filter configurations on both the Cisco Nexus switches and the access switches after upgrading to the latest build.
  • Page 121 2016 Aug 29 19:14:19 TOR9-leaf4 %$ VDC-1 %$ %STP-2-L2GW_BACKBONE_BLOCK: L2 Gateway Backbone port inconsistency blocking port Ethernet1/1 on MST0000. 2016 Aug 29 19:14:19 TOR9-leaf4 %$ VDC-1 %$ %STP-2-L2GW_BACKBONE_BLOCK: L2 Gateway Backbone port inconsistency blocking port port-channel13 on MST0000. switch# show spanning-tree Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 122 BPDUs from the access switches. In that case, the access ports on VTEPs lose the advantage of rapid transmission, instead forwarding on Ethernet segment link flap. (They have to go through a proposal and agreement handshake before assuming the FWD-Desg role). Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 123: Configuring Vxlan Evpn Multihoming Traffic Flows

    If the ESI link at L1 fails, there is no path for the bridged traffic to reach from H1 to H2 except via the overlay. Therefore, the local bridged traffic takes the sub-optimal path, similar to the H1 to H3 orphan flow. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 124 If switch L1 gets isolated from the core, it must not continue to attract access traffic, as it will not be able to encapsulate and send it on the overlay. This means that the access links must be brought down at L1 if L1 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 125 Instead, H3 shows up as a remote host in the IP table at L1, installed in the context of L3 VNI. This packet must be encapsulated in the router-MAC of L2 and routed to L2 via VXLAN overlay. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 126 Figure 17: L1 is Distributed Anycast Gateway. H1, H2, and H3 are in different VLANs. H1->H3 routing happens via VXLAN tunnel encapsulation. In VPC, H3 ARP would have been synced via MCT and direct routing. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 127 If switch L1 gets isolated from the core, it must not continue to attract access traffic, as it will not be able to encapsulate and send it on the overlay. It means that the access links must be brought down at L1 if L1 loses core reachability. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 128: Evpn Multihoming Remote Traffic Flows

    This section describes how the ECMP is achieved at switch L3 for both bridged and routed cases and how the system interacts with core and access failures. Figure 20: Layer 2 VXLAN Gateway. L3 performs MAC ECMP to L1/L2. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 129 MAC-IP Route remains the same as used in the current vPC multihoming and standalone single-homing solutions. However, now it has a non-zero ESI field that indicates that this is a multihomed host and it is a candidate for ECMP Path Resolution. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 130 ECMP list for the given ES. Figure 21: Layer 2 VXLAN Gateway. ESI failure on L1. L3 withdraws L1 from MAC ECMP list. This will happen due to EAD/ES mass withdrawal from L1. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 131 L1 and L2 advertise the MAC-IP route for Host H2. Due to the receipt of these routes, L3 builds an L3 ECMP list comprising of L1 and L2. Figure 23: Layer 3 VXLAN Gateway. L3 does IP ECMP to L1/L2 for inter subnet traffic. Access Failure for Remote Routed Traffic Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 132 Figure 24: Layer 3 VXLAN Gateway. ESI failure causes ES mass withdrawal that only impacts L2 ECMP. L3 ECMP continues until Type2 is withdrawn. L3 traffic reaches H2 via suboptimal path L3->L1->L2 until then. Core Failure for Remote Routed Traffic Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 133: Evpn Multihoming Bum Flows

    Figure 26: BUM traffic originating at L3. L2 is the DF for ES1 and ES2. L2 decapsulates and forwards to ES1, ES2 and orphan. L1 decapsulates and only forwards to orphan. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 134 Figure 27: BUM traffic originating at L1. L2 is the DF for ES1 and ES2. However, L2 must perform split horizon check here as it shares ES1 and ES2 with L1. L2 however Ethernet Segment Route (Type 4) Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 135 DF. Since L2 is the only TOR left in the Ordinal Table, it takes over DF role for all VLANs. BGP EVPN multihoming on Cisco Nexus 9000 Series switches provides minimum operational and cabling expenditure, provisioning simplicity, flow based load balancing, multi pathing, and fail-safe redundancy.

  • Page 136: Configuring Vlan Consistency Checking

    CFS capable switches in the network and to discover the feature capabilities in all the CFS capable switches. You can use CFS over IP (CFSoIP) to distribute and synchronize a configuration on one Cisco device or with all other Cisco devices in your network.

  • Page 137: Configuring Vlan Consistency Checking

    The show nve ethernet-segment command now displays the following details: • The list of VLANs for which consistency check is failed. • Remaining value (in seconds) of the global VLAN CC timer. switch# sh nve ethernet-segment ESI Database Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 138: Configuring Esi Arp Suppression

    ESI ARP suppression is a per-VNI (L2-VNI) feature. ESI ARP suppression is supported in both L2 (no SVI) and L3 modes. Beginning with Cisco NX-OS Release 7.0(3)I5(2), only L3 mode is supported. The ESI ARP suppression cache is built by: •...

  • Page 139: Limitations For Esi Arp Suppression

    Limitations for ESI ARP Suppression See the following limitations for ESI ARP suppression: • ESI multihoming solution is supported only on Cisco Nexus 9300 Series switches at the leafs. • ESI ARP suppression is only supported in L3 [SVI] mode.
  • Page 140 Multihoming DEL error invalid current state:0 Peer sync DEL error MAC mismatch Peer sync DEL error second delete Peer sync DEL error deleteing TL route True local DEL error deleteing PS RO route :0 switch# Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 141: Vxlan Bud Node Over Vpc

    A P P E N D I X VXLAN Bud Node Over VPC • VXLAN Bud Node Over VPC Overview, page 130 • VXLAN Bud Node Over VPC Topology Example, page 131 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 142: Vxlan Bud Node Over Vpc Overview

    VLAN. This SVI should have proxy ARP enabled. For example: Interface Vlan2 ip proxy-arp For Cisco Nexus 9200 Series switches and Cisco Nexus 9300-EX switches, use the system nve infra-vlans Note command to configure any VLANs that are used as infra-VLANs. VLANs that are configured without a vn-segment are considered infra-VLANs.

  • Page 143: Vxlan Bud Node Over Vpc Topology Example

    10.200.1.252/24 no ipv6 redirects ip router ospf 1 area 0.0.0.0 ip pim sparse-mode hsrp version 2 hsrp 1 ip 10.200.1.254 • Route-maps for matching multicast groups Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 144 10002 mcast-group 225.1.1.1 member vni 10003 mcast-group 225.1.1.1 • Loopback interface configuration interface loopback0 ip address 101.101.101.101/32 ip address 99.99.99.99/32 secondary ip router ospf 1 area 0.0.0.0 ip pim sparse-mode Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 145 Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary, operational primary Number of vPCs configured Peer Gateway : Enabled Dual-active excluded VLANs Graceful Consistency Check : Enabled Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 146 Up, 99.99.99.99, DP Up, 99.99.99.99, DP IP, Host Reach Mode Nve Vni Configuration 10001-10003 10001-10003 Interface-vlan admin up 2,2000 2,2000 Interface-vlan routing 1-4,2000 1-4,2000 capability Allowed VLANs 1-4,101-103,2000 1-4,101-103,2000 Local suspended VLANs Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 147: Dhcp Relay In Vxlan Bgp Evpn

    The relay agent adds all of the appropriate sub-options and then forwards the renew and release request packets to the original DHCP server. For this function, Cisco’s proprietary implementation is sub-option 152(0x98). You can use the ip dhcp relay sub-option type cisco command to manage the function.

  • Page 148: Dhcp Relay In Vxlan Bgp Evpn Example

    IP address so that DHCP messages are able to be forwarded over the network. For this function, Cisco’s proprietary implementation is sub-option 150(0x96). You can use the ip dhcp relay sub-option type cisco command to manage the function.

  • Page 149: Basic Vxlan Bgp Evpn Configuration

    192.1.33.2/24 ip router ospf 1 area 0.0.0.0 ip pire sparse-mode no shutdown interface loopback0 ip address 1.1.1.1/32 ip router ospf 1 area 0.0.0.0 ip pim sparse—mode interface loopbackl Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 150 900001 associate—vrf member vni 2001001 mcast—group 225.4.0.1 interface Ethernetl/49 switchport mode trunk switchport trunk alluwed vlan 10,1001 spanning—tree port type edge trunk Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 151: Dhcp Relay On Vteps

    Put DHCP server (192.1.42.3) into the default VRF and make sure it is reachable from both 9372-1 and 9372-2 through the default VRF. 9372-1# sh run int vl 10 !Command: show running-config interface Vlan10 !Time: Mon Aug 24 07:51:16 2015 version 7.0(3)I1(3) Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 152 Vlanl001 ip dhcp relay address 192.1.42.3 use—vrf default Debug Output • The following is a packet dump for DHCP interact sequences. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 153 Agent Remote ID: f8c2882333a5 Option 82 Suboption: (151) VRF name/VPN ID Option 82 Suboption: (11) Server ID Override Length: 4 Server ID Override: 172.16.16.1 (172.16.16.1) Option 82 Suboption: (5) Link selection Length: 4 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 154: Client On Tenant Vrf (Svi X) And Server On The Same Tenant Vrf (Svi Y)

    Put DHCP server (192.1.42.3) into VRF of vxlan-900001 and make sure it is reachable from both 9372-1 and 9372-2 through VRF of vxlan-900001. 9372-1# sh run int vl 10 !Command: show running-config interface Vlan10 !Time: Mon Aug 24 09:10:26 2015 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 155 !Command: show running-config dhcp !Time: Mon Aug 24 08:26:00 2015 version 7.0(3)11(3) feature dhcp service dhcp ip dhcp relay ip dhcp relay information option I4ip dhcp relay information option vpn ipv6 dhcp relay Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 156 • DHCP Discover packet 9372-1 sent to DHCP server. giaddr is set to 11.11.11.11(loopback1) and suboptions 5/11/151 are set accordingly. Bootstrap Protocol Message type: Boot Request (1) Hardware type: Ethernet (0x01) Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 157 65535 (evpn)segid: 900001 tunnelid: 0x2020202 encap: VXLAN 172.16.16.11/32, ubest/mbest: 1/0, attached *via 172.16.16.11, Vlan1001, [190/0], 00:13:56, hmm 192.1.42.0/24, ubest/mbest: 1/0, attached *via 192.1.42.1, Vlan10, [0/0], 00:36:08, direct 192.1.42.1/32, ubest/mbest: 1/0, attached Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 158: Client On Tenant Vrf (Vrf X) And Server On Different Tenant Vrf (Vrf Y)

    64 bytes from 192.1.42.3: icmp_seq=0 ttl=253 time=0.678 ms - 192.1.42.3 ping statistics - 1 packets transmitted, 1 packets received, 0.00% packet loss round-trip min/avg/max = 0.678/0.678/0.678 ms DHCP Relay Configuration Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 159 20150825 08:59:37.760733 33.33.33.33 -> 192.1.42.3 DHCP DHCP Request - Transaction ID 0x3eebccae 20150825 08:59:37.761297 192.1.42.3 -> 33.33.33.33 DHCP DHCP ACK - Transaction ID 0x3eebccae 20150825 08:59:37.761554 172.16.16.1 -> 172.16.16.11 DHCP DHCP ACK - Transaction ID 0x3eebccae Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 160: Client On Tenant Vrf And Server On Non-Default Non-Vxlan Vrf

    = 1.024/1.024/1.024 ms 9372-2# sh run int m0 !Command: show running-config interface mgmt0 !Time: Tue Aug 25 09:17:47 2015 version 7.0(3)I1(3) interface mgmt0 vrf member management ip address 10.122.165.148/25 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 161 20150825 09:30:56.216931 0.0.0.0 -> 255.255.255.255 DHCP DHCP Request - Transaction ID 0x28a8606d 20150825 09:30:56.218426 172.16.16.1 -> 172.16.16.11 DHCP DHCP ACK - Transaction ID 0x28a8606d 9372-1# ethanalyzer local interface mgmt display-filter "ip.src==10.122.164.147 or ip.dst==10.122.164.147" limit-captured-frames 0 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 162: Configuring Vpc Peers Example

    Link selection: 172.16.16.0 (172.16.16.0) Configuring VPC Peers Example The following is an example of how to configure routing between VPC peers in the overlay VLAN for a DHCP relay configuration. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 163 /* Only required for VPC VTEP. */ • Advertise LoX into the Layer 3 VRF BGP. Router bgp 2 vrf X network 10.1.1.42/32 • Configure DHCP relay on the SVI under the VRF. interface Vlan1601 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 164: Vpc Vtep Dhcp Relay Configuration Example

    GiAddr. The following are examples of these configurations: • Configuration of SVI within underlay routing: /* vPC Peer-1 */ router ospf UNDERLAY vrf tenant-vrf interface Vlan2000 no shutdown mtu 9216 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 165 192.168.1.2/30 192.168.1.1 /* vPC Peer-2 */ interface Vlan2000 no shutdown mtu 9216 vrf member tenant-vrf ip address 192.168.1.2/30 vrf context tenant-vrf ip route 192.168.1.1/30 192.168.1.2 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 166 DHCP Relay in VXLAN BGP EVPN vPC VTEP DHCP Relay Configuration Example Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 167: Evpn With Transparent Firewall Insertion

    However, by default, VXLAN EVPN requires a distributed anycast gateway on all LEAFs. To address the Layer 2 transparent firewall requirement with VXLAN EVPN, a special topology can be used. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 168 All TOR leafs have a Layer 2 VNI VLAN X. There is no SVI for VLAN X. The service leafs that are connected to the firewall have Layer 2 VNI VLAN X, non-VXLAN VLAN Y, and SVI Y with a HSRP gateway. Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 169: Evpn With Transparent Firewall Insertion Example

    VXLAN only supports an anycast gateway, not a centralized gateway. EVPN with Transparent Firewall Insertion Example • Host in VLAN X: 10.0.94.101 Note • TOR Leaf: N9372-1 • Service Leaf in vPC: N9332-1 and N9332-2 • Border Leaf: N9332-5 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 170 10.0.94.2/24 hsrp 0 preempt priority 255 ip 10.0.94.1 interface nve1 member vni 100094 mcast-group 239.1.1.1 router bgp 64500 routerid 1.1.2.1 neighbor 1.1.1.1 remote-as 64500 address-family l2vpn evpn Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 171 Ten-1 address-family ipv4 unicast network 10.0.94.0/24 /*advertise /24 for SVI 95 subnet; it is not VXLAN anymore*/ advertise l2vpn evpn evpn vni 100094 l2 rd auto Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 172: Show Command Examples

    N93965# sh ip route 10.0.94.101 IP Route Table for VRF "default" '*' denotes best ucast nexthop '**' denotes best mcast nexthop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 173 EVPN with Transparent Firewall Insertion Show Command Examples 10.0.94.0/24, ubest/mbest: 1/0 *via 10.100.5.0, [20/0], 03:14:27, bgp65000,external, tag 6450 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 174 EVPN with Transparent Firewall Insertion Show Command Examples Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...

  • Page 175: Ipv6 Across A Vxlan Evpn Fabric

    VXLAN fabric through the L2VPN EVPN address family as EVPN route-type 2 or 5. These routes are advertised as EVPN routes on the SPINE. Note Configuring IPv6 Across a VXLAN EVPN Fabric Example Topology for the example: Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 176 10 name RED vn-segment 10010 • Configure the VLAN for L3 VNI . vlan 100 name RED_L3_VNI_VLAN vn-segment 20010 • Define the anycast gateway MAC. fabric forwarding anycast-gateway-mac 0000.2222.3333 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 177 10.1.1.1/24 ipv6 address 2001::1/64 fabric forwarding mode anycast-gateway Note IPv6 ND suppression is not supported on Cisco Nexus 9000 Series switches. (7.0(3)I3(1) and earlier releases) • Configure SVI definition for VLAN 100. interface Vlan100 description RED_L3_VNI_VLAN...

  • Page 178: Show Command Examples

    678 Paths: (1 available, best #1) Flags: (0x00010a) on xmit-list, is not in l2rib/evpn Advertised path-id 1 Path type: local, path is valid, is best path, no labeled nexthop Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 179 • Check the L2ROUTE and ensure that the MAC-IP was learned on the remote VTEP - 9396-A-VTEP. rswV1leaf14# show l2route evpn mac-ip evi 1413 host-ip 2001::64 Mac Address Prod Host IP Next Hop (s) -------------- ---- --------------------------------------- -------------- 7c69.f614.2bc1 BGP 2001::64 198.19.0.15 Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 180 IPv6 Across a VXLAN EVPN Fabric Show Command Examples Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x...
  • Page 181 14, 55, 67 host-reachability protocol bgp 59, 63, 65 how interface rd auto 61, 65 retain route-target all route-map permitall out Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x IN-1...
  • Page 182 41, 42, 87 show nve vni vrf context 41, 42, 59, 87 59, 61, 63 show nve vni ingress-replication 41, 42 vrf member show nve vni summary Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x IN-2...

Table of Contents