Page 1
H3C S5560X-EI Switch Series VXLAN Configuration Guide New H3C Technologies Co., Ltd. http://www.h3c.com.hk Software version: Release 1110 series Document version: 6W102-20171207...
Page 2
, H3CS, H3CIE, H3CNE, Aolynk, Care, , IRF, NetPilot, Netflow, SecEngine, SecPath, SecCenter, SecBlade, Comware, ITCMM and HUASAN are trademarks of New H3C Technologies Co., Ltd. All other trademarks that may be mentioned in this manual are the property of their respective owners Notice The information in this document is subject to change without notice.
Page 3
Preface This configuration guide describes the VXLAN fundamentals and configuration procedures. This preface includes the following topics about the documentation: • Audience • Conventions • Obtaining documentation • Technical support • Documentation feedback Audience This documentation is intended for: • Network planners.
Page 4
Convention Description Folder. Symbols Convention Description An alert that calls attention to important information that if not understood or followed WARNING! can result in personal injury. An alert that calls attention to important information that if not understood or followed CAUTION: can result in data loss, data corruption, or damage to hardware or software.
Page 5
Obtaining documentation To access the most up-to-date H3C product documentation, go to the H3C website at http://www.h3c.com.hk To obtain information about installation, configuration, and maintenance, click http://www.h3c.com.hk/Technical_Documents...
The transport edge devices are VXLAN tunnel endpoints (VTEP). An H3C VTEP uses VSIs and VXLAN tunnels to provide VXLAN services. • VSI—A virtual switch instance is a virtual Layer 2 switched domain. Each VSI provides switching services only for one VXLAN.
Figure 1 VXLAN network model VSI/VXLAN 10 VSI/VXLAN 10 VSI/VXLAN 20 VSI/VXLAN 20 Terminal VXLAN tunnel Terminal Terminal Terminal Overlay network VTEP 2 VTEP 1 Site 2 Transport network Site 1 Underlay network VXLAN packet format As shown in Figure 2, a VTEP encapsulates a frame in the following headers: •...
Working mechanisms The VTEP uses the following process to forward an inter-site frame: Assigns the frame to its matching VXLAN if the frame is sent between sites. Performs MAC learning on the VXLAN's VSI. Forwards the frame through VXLAN tunnels. This section describes this process in detail.
• Local MAC—MAC entries dynamically learned from the local site. The outgoing interfaces for the MAC address entries are site-facing interfaces. VXLAN does not support static local-MAC entries. • Remote MAC—MAC entries learned from a remote site. The outgoing interfaces for the MAC address entries are VXLAN tunnel interfaces.
Page 11
Figure 4 Inter-site unicast Flood The VTEP floods a broadcast, multicast, or unknown unicast frame to all site-facing interfaces and VXLAN tunnels in the VXLAN, except for the incoming interface. VXLAN supports the unicast mode for flood traffic. The mode is also called head-end replication. The source VTEP replicates the flood frame, and then sends one replica to the destination IP address of each VXLAN tunnel in the VXLAN.
Access mode of VSIs The VTEP supports VLAN access mode for VSIs. In VLAN access mode, Ethernet frames received from or sent to the local site must contain 802.1Q VLAN tags. • For an Ethernet frame received from the local site, the VTEP removes all its 802.1Q VLAN tags before forwarding the frame.
VTEP 1 de-encapsulates the ARP reply, creates a suppression entry for Terminal 7, and forwards the ARP reply to Terminal 1. Terminal 4 sends an ARP request to obtain the MAC address of Terminal 1 or Terminal 7. VTEP 1 creates a suppression entry for Terminal 4 and replies to the ARP request. Terminal 10 sends an ARP request to obtain the MAC address of Terminal 1.
Configuring basic VXLAN features Configuration restrictions and guidelines Before you configure VXLAN settings, perform the following tasks: Execute the switch-mode 1 command in system view to set the system operating mode to VXLAN. Save the running configuration to the next-startup configuration file. Reboot the device.
Setting the forwarding mode for VXLANs Step Command Remarks Enter system view. system-view By default, Layer 3 forwarding is enabled for VXLANs. • Enable Layer 2 If the VTEP is not a VXLAN IP gateway, forwarding: enable Layer 2 forwarding for VXLANs. If undo vxlan the VTEP is a VXLAN IP gateway, enable Enable Layer 2 or Layer...
Step Command Remarks By default, no global source address is specified for VXLAN tunnels. Specify a global tunnel global source address for A VXLAN tunnel uses the global source source-address ip-address VXLAN tunnels. address if you do not specify a source interface or source address for the tunnel.
Mapping a static Ethernet service instance to a VSI A static Ethernet service instance matches a list of VLANs on a site-facing interface. The VTEP assigns customer traffic from the VLANs to a VXLAN by mapping the Ethernet service instance to a VSI.
• A remote AAA server issues an authorization VSI to the user. For a dynamic Ethernet service instance to match traffic by source MAC address, enable MAC-based traffic match mode for the dynamic Ethernet service instance. To enable MAC-based traffic match mode for dynamic Ethernet service instances on an interface: Step Command Remarks...
Step Command Remarks By default, remote-MAC address learning is enabled. When network attacks occur, disable Disable remote-MAC vxlan tunnel mac-learning remote-MAC address learning to prevent address learning. disable the device from learning incorrect remote MAC addresses. You can manually add static remote-MAC address entries.
Setting the destination UDP port number of VXLAN packets Step Command Remarks Enter system view. system-view By default, the destination UDP port number is 4789 for VXLAN packets. You must configure the same destination UDP port number on all VTEPs in a VXLAN. If you modify the destination UDP port Set a destination UDP vxlan udp-port port-number...
As a best practice, disable remote ARP learning for VXLANs only when the controller and VTEPs are synchronizing entries. To disable remote ARP learning for VXLANs: Step Command Remarks Enter system view. system-view Disable remote ARP By default, remote ARP learning is vxlan tunnel arp-learning disable learning for VXLANs.
Figure 7 Network diagram Configuration procedure Set the system operation mode to VXLAN on Switch A, Switch B, and Switch C. This step uses Switch A as an example. <SwitchA> system-view [SwitchA] switch-mode 1 Reboot device to make the configuration take effect. [SwitchA] quit <SwitchA>...
Page 23
[SwitchA-Loopback0] quit # Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 1. [SwitchA] interface tunnel 1 mode vxlan [SwitchA-Tunnel1] source 1.1.1.1 [SwitchA-Tunnel1] destination 2.2.2.2 [SwitchA-Tunnel1] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 2. [SwitchA] interface tunnel 2 mode vxlan [SwitchA-Tunnel2] source 1.1.1.1 [SwitchA-Tunnel2] destination 3.3.3.3...
Page 24
[SwitchB-Tunnel2] destination 1.1.1.1 [SwitchB-Tunnel2] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3. [SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source 2.2.2.2 [SwitchB-Tunnel3] destination 3.3.3.3 [SwitchB-Tunnel3] quit # Assign Tunnel 2 and Tunnel 3 to VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan-10] tunnel 2...
[SwitchC-Tunnel3] source 3.3.3.3 [SwitchC-Tunnel3] destination 2.2.2.2 [SwitchC-Tunnel3] quit # Assign Tunnel 1 and Tunnel 3 to VXLAN 10. [SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan-10] tunnel 1 [SwitchC-vsi-vpna-vxlan-10] tunnel 3 [SwitchC-vsi-vpna-vxlan-10] quit [SwitchC-vsi-vpna] quit # On GigabitEthernet 1/0/1, create Ethernet service instance 1000 to match VLAN 2. [SwitchC] interface gigabitethernet 1/0/1 [SwitchC-GigabitEthernet1/0/1] port link-type trunk [SwitchC-GigabitEthernet1/0/1] port trunk permit vlan 2...
Page 26
Multicast Restrain Unknown Unicast Restrain: - MAC Learning : Enabled MAC Table Limit MAC Learning rate Drop Unknown Flooding : Enabled VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flood proxy Tunnel1 0x5000001 Manual Disabled Tunnel2 0x5000002 Manual Disabled ACs:...
Configuring VXLAN IP gateways Overview The following are available IP gateway placement designs for VXLANs: • VXLAN IP gateways separated from VTEPs—Use a VXLAN-unaware device as a gateway to the external network for VXLANs. On the gateway, you do not need to configure VXLAN settings.
Figure 8 VXLAN IP gateway separated from VTEPs Centralized VXLAN IP gateway deployment As shown in Figure 9, a VTEP acts as a gateway for user terminals in the VXLANs. The VTEP both terminates the VXLANs and performs Layer 3 forwarding for the user terminals. Figure 9 Centralized VXLAN IP gateway placement design Transport network...
VTEP 3 de-encapsulates the ARP request, creates an ARP entry for the user terminal, and sends an ARP reply to the user terminal. VTEP 1 forwards the ARP reply to the user terminal. The user terminal learns the MAC address of the gateway, and sends the Layer 3 traffic to the gateway.
Page 30
Figure 11 Distributed VXLAN IP gateway placement design L3 network VTEP/Border gateway VTEP/Distributed VXLAN IP gateway VTEP/Distributed VTEP/Distributed VXLAN tunnel VXLAN tunnel VXLAN IP VXLAN IP gateway gateway VTEP Terminal Terminal Terminal Terminal Terminal Terminal Site 1 Site 2 Site 3 Site 4 Site 5 Site 6...
Page 31
Intra-VXLAN traffic forwarding between sites As shown in Figure 12, the network uses the following process to forward traffic in a VXLAN between sites (for example, from Terminal 1 to Terminal 4 in VXLAN 10): Terminal 1 sends an ARP request to obtain the MAC address of Terminal 4. GW 1 performs the following operations: a.
b. Replaces the sender MAC address in the request with the MAC address of VSI-interface 20 on GW 2, and then sends the reply to GW 1. GW 1 de-encapsulates the ARP request and creates an ARP entry for Terminal 5. The entry contains Terminal 5's IP address 20.1.1.12, the MAC address of VSI-interface 20 on GW 2, and the incoming tunnel interface.
Configuring a distributed VXLAN IP gateway Configuration restrictions and guidelines Make sure a VSI interface uses the same MAC address to provide service on distributed VXLAN IP gateways connected to IPv4 sites. Make sure a VSI interface uses different link-local addresses to provide service on distributed VXLAN IP gateways connected to both IPv4 and IPv6 sites.
Step Command Remarks By default, dynamic ARP entry synchronization is disabled for distributed VXLAN IP gateways. When local proxy ARP is enabled on distributed VXLAN IP gateways, each gateway learns ARP information (Optional.) Enable independently. A gateway does not forward dynamic ARP entry arp distributed-gateway ARP packets destined for it to other...
Step Command Remarks Restore the default default settings on the interface. Bring up the interface. undo shutdown By default, a VSI interface is up. Displaying and maintaining VXLAN IP gateway Execute display commands in any view and reset commands in user view. Task Command Display information about VSI...
Page 36
<SwitchA> system-view [SwitchA] switch-mode 1 Reboot device to make the configuration take effect. [SwitchA] quit <SwitchA> reboot Start to check configuration with next startup configuration file, please wait..DONE! Current configuration may be lost after the reboot, save current configuration? [Y/N]:y This command will reboot the device.
Page 37
[SwitchA-vsi-vpna-vxlan-10] quit [SwitchA-vsi-vpna] quit # On GigabitEthernet 1/0/1, create Ethernet service instance 1000 to match VLAN 2. [SwitchA] interface gigabitethernet 1/0/1 [SwitchA-GigabitEthernet1/0/1] port link-type trunk [SwitchA-GigabitEthernet1/0/1] port trunk permit vlan 2 [SwitchA-GigabitEthernet1/0/1] service-instance 1000 [SwitchA-GigabitEthernet1/0/1-srv1000] encapsulation s-vid 2 # Map Ethernet service instance 1000 to VSI vpna. [SwitchA-GigabitEthernet1/0/1-srv1000] xconnect vsi vpna [SwitchA-GigabitEthernet1/0/1-srv1000] quit [SwitchA-GigabitEthernet1/0/1] quit...
Page 38
# Specify VSI-interface 1 as the gateway interface for VSI vpna. [SwitchB] vsi vpna [SwitchB-vsi-vpna] gateway vsi-interface 1 [SwitchB-vsi-vpna] quit Configure Switch C: # Enable L2VPN. <SwitchC> system-view [SwitchC] l2vpn enable # Enable Layer 2 forwarding for VXLANs. [SwitchC] undo vxlan ip-forwarding # Create VSI vpna and VXLAN 10.
Page 39
Verifying the configuration Verify the VXLAN IP gateway settings on Switch B: # Verify that the VXLAN tunnel interfaces are up on Switch B. [SwitchB] display interface tunnel 2 Tunnel2 Current state: UP Line protocol state: UP Description: Tunnel2 Interface Bandwidth: 64 kbps Maximum transmission unit: 64000 Internet protocol processing: Disabled...
MAC Learning rate Drop Unknown Flooding : Enabled Gateway interface : VSI-interface 1 VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flood proxy Tunnel2 0x5000002 Manual Disabled Tunnel3 0x5000003 Manual Disabled # Verify that Switch B has created ARP entries for the user terminals. [SwitchB] display arp Type: S-Static D-Dynamic...
Page 42
[SwitchA-vsi-vpna] quit # Create VSI vpnc and VXLAN 30. [SwitchA] vsi vpnc [SwitchA-vsi-vpnc] vxlan 30 [SwitchA-vsi-vpnc-vxlan-30] quit [SwitchA-vsi-vpnc] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch B and Switch C. [SwitchA] interface loopback 0 [SwitchA-Loopback0] ip address 1.1.1.1 255.255.255.255 [SwitchA-Loopback0] quit...
Page 43
[SwitchA-Vsi-interface1] ip address 10.1.1.1 255.255.255.0 [SwitchA-Vsi-interface1] mac-address 1-1-1 # Specify VSI-interface 1 as a distributed gateway and enable local proxy ARP on the interface. [SwitchA-Vsi-interface1] distributed-gateway local [SwitchA-Vsi-interface1] local-proxy-arp enable [SwitchA-Vsi-interface1] quit # Create VSI-interface 2 and assign the interface an IP address and a MAC address. The IP address will be used as the gateway address for VXLAN 30.
Page 44
[SwitchB] interface tunnel 2 mode vxlan [SwitchB-Tunnel2] source 2.2.2.2 [SwitchB-Tunnel2] destination 1.1.1.1 [SwitchB-Tunnel2] quit # Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3. [SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source 2.2.2.2 [SwitchB-Tunnel3] destination 3.3.3.3 [SwitchB-Tunnel3] quit # Assign Tunnel 2 to VXLAN 10.
Page 45
[SwitchC] vsi vpnb [SwitchC-vsi-vpnb] vxlan 30 [SwitchC-vsi-vpnb-vxlan-30] quit [SwitchC-vsi-vpnb] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch B. [SwitchC] interface loopback 0 [SwitchC-Loopback0] ip address 3.3.3.3 255.255.255.255 [SwitchC-Loopback0] quit # Create a VXLAN tunnel to Switch A.
Page 46
# Specify VSI-interface 1 as a distributed gateway and enable local proxy ARP on the interface. [SwitchC-Vsi-interface1] distributed-gateway local [SwitchC-Vsi-interface1] local-proxy-arp enable [SwitchC-Vsi-interface1] quit # Enable dynamic ARP entry synchronization for distributed VXLAN IP gateways. [SwitchC] arp distributed-gateway dynamic-entry synchronize # Specify VSI-interface 1 as the gateway interface for VSI vpna.
Page 47
Line protocol state: UP Description: Vsi-interface1 Interface Bandwidth: 1000000 kbps Maximum transmission unit: 1500 Internet address: 10.1.1.1/24 (primary) IP packet frame type: Ethernet II, hardware address: 0001-0001-0001 IPv6 packet frame type: Ethernet II, hardware address: 0001-0001-0001 Physical: Unknown, baudrate: 1000000 kbps Last clearing of counters: Never Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec...
Page 48
[SwitchB] display interface tunnel 2 Tunnel2 Current state: UP Line protocol state: UP Description: Tunnel2 Interface Bandwidth: 64 kbps Maximum transmission unit: 64000 Internet protocol processing: Disabled Last clearing of counters: Never Tunnel source 2.2.2.2, destination 1.1.1.1 Tunnel protocol/transport UDP_VXLAN/IP Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops...
Gateway interface : VSI-interface 1 VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flood proxy Tunnel2 0x5000002 Manual Disabled # Verify that Switch B has created ARP entries for the user terminals. [SwitchB] display arp Type: S-Static D-Dynamic O-Openflow R-Rule...
Page 50
Figure 15 Network diagram Configuration procedure Set the system operation mode to VXLAN on Switch A, Switch B, and Switch C. This step uses Switch A as an example. <SwitchA> system-view [SwitchA] switch-mode 1 Reboot device to make the configuration take effect. [SwitchA] quit <SwitchA>...
Page 51
# Create VSI vpnb and VXLAN 20. [SwitchA] vsi vpnb [SwitchA-vsi-vpnb] vxlan 20 [SwitchA-vsi-vpnb-vxlan-20] quit [SwitchA-vsi-vpnb] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch B and Switch C. [SwitchA] interface loopback 0 [SwitchA-Loopback0] ip address 1.1.1.1 255.255.255.255 [SwitchA-Loopback0] quit...
Page 52
[SwitchA] interface vsi-interface 1 [SwitchA-Vsi-interface1] ipv6 address 1::1/64 anycast # Specify VSI-interface 1 as a distributed gateway and enable local ND proxy on the interface. [SwitchA-Vsi-interface1] distributed-gateway local [SwitchA-Vsi-interface1] local-proxy-nd enable [SwitchA-Vsi-interface1] quit # Specify VSI-interface 1 as the gateway interface for VSI vpna. [SwitchA] vsi vpna [SwitchA-vsi-vpna] gateway vsi-interface 1 [SwitchA-vsi-vpna] quit...
Page 53
# Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3. [SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source 2.2.2.2 [SwitchB-Tunnel3] destination 3.3.3.3 [SwitchB-Tunnel3] quit # Assign Tunnel 2 and Tunnel 3 to VXLAN 10. [SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan-10] tunnel 2 [SwitchB-vsi-vpna-vxlan-10] tunnel 3...
Page 54
[SwitchC-vsi-vpnb-vxlan-20] quit [SwitchC-vsi-vpnb] quit # Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch B. [SwitchC] interface loopback 0 [SwitchC-Loopback0] ip address 3.3.3.3 255.255.255.255 [SwitchC-Loopback0] quit # Create a VXLAN tunnel to Switch A.
Page 55
[SwitchC-Vsi-interface1] distributed-gateway local [SwitchC-Vsi-interface1] local-proxy-nd enable [SwitchC-Vsi-interface1] quit # Specify VSI-interface 1 as the gateway interface for VSI vpna. [SwitchC] vsi vpna [SwitchC-vsi-vpna] gateway vsi-interface 1 [SwitchC-vsi-vpna] quit # Create VSI-interface 2 and assign the interface an IPv6 anycast address. The IP address will be used as the gateway address for VXLAN 20.
Page 56
Internet protocol processing: Disabled IP packet frame type: Ethernet II, hardware address: 0011-2200-0102 IPv6 packet frame type: Ethernet II, hardware address: 0011-2200-0102 Physical: Unknown, baudrate: 1000000 kbps Last clearing of counters: Never Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops Output: 0 packets, 0 bytes, 0 drops...
Page 57
VXLAN ID : 20 Tunnels: Tunnel Name Link ID State Type Flood proxy Tunnel1 0x5000001 Manual Disabled Tunnel2 0x5000002 Manual Disabled # Verify that Switch A has created neighbor entries for the user terminals. [SwitchA] display ipv6 neighbors all Type: S-Static D-Dynamic O-Openflow R-Rule...
Page 58
Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops Output: 0 packets, 0 bytes, 0 drops # Verify that the VSI interfaces are up. [SwitchB] display interface Vsi-interface 1 Vsi-interface1 Current state: UP Line protocol state: UP Description: Vsi-interface1 Interface Bandwidth: 1000000 kbps...
Page 59
1::100 0001-0000-0047 N/A Vsi1 STALE D 4::400 0002-0000-0047 N/A Vsi2 REACH D FE80::201:FF:FE00:0 0001-0000-0000 N/A Vsi1 STALE D FE80::201:FF:FE00:47 0001-0000-0047 N/A Vsi1 STALE D FE80::202:FF:FE00:0 0002-0000-0000 N/A Vsi2 STALE D # Verify that Switch B has created FIB entries for the user terminals. [SwitchB] display ipv6 fib 1::100 Destination count: 1 FIB entry count: 1 Flag:...