Dell EMC VxRail P series Deployment Manual
Multirack deployment of vxrail cluster using network virtualization overlays (nvo)
Hide thumbs
Also See for VxRail P series:
- Network manual (48 pages) ,
- Owner's manual (33 pages) ,
- Owner's manual (48 pages)
Table of Contents
Advertisement
Dell EMC VxRail Multirack Deployment Guide
Multirack deployment of VxRail cluster using Network Virtualization
Overlays (NVO)
Dell EMC Configuration and Deployment Guide
Abstract
This document provides step-by-step deployment instructions for Dell
EMC Networking OS10EE static VXLAN tunnels in an OSPF routed
environment. This provides the foundation that is needed for multirack
VxRail host discovery and deployment in a modern data center.
February 2019
Advertisement
Table of Contents
Related Manuals for Dell EMC VxRail P series
Summary of Contents for Dell EMC VxRail P series
- Page 1 Dell EMC VxRail Multirack Deployment Guide Multirack deployment of VxRail cluster using Network Virtualization Overlays (NVO) Abstract This document provides step-by-step deployment instructions for Dell EMC Networking OS10EE static VXLAN tunnels in an OSPF routed environment. This provides the foundation that is needed for multirack VxRail host discovery and deployment in a modern data center.
-
Page 2: Revisions
Use, copying, and distribution of any software described in this publication requires an applicable software license. © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. -
Page 3: Table Of Contents
Supported switches and operating systems ....................... 8 Typographical conventions ..........................8 Attachments ................................ 8 Hardware overview ............................... 9 Dell EMC VxRail P570 ............................9 Dell EMC VxRail E560 ............................9 Dell EMC Networking Z9264F-ON ........................10 Dell EMC Networking S3048-ON ........................10 Topology .................................. - Page 4 VxRail deployment values ............................48 Connecting a workstation or laptop for VxRail initialization ..................49 Validated components ..............................51 Dell EMC Networking switches ......................... 51 VxRail E560 nodes ............................51 VxRail P570 nodes ............................51 Dell EMC VxRail Multirack Deployment Guide...
- Page 5 VxRail appliance software ..........................52 Technical resources ..............................53 G Support and feedback ..............................54 Dell EMC VxRail Multirack Deployment Guide...
-
Page 6: Introduction
L2 traffic between separate subnets. Figure 1 shows Dell EMC Networking switches in Virtual Link Trunking (VLT) pairs connected over an OSPF enabled IP underlay using VXLAN tunnels. This topology enables the creation of multiple virtual networks over one common IP underlay network. -
Page 7: Objective
Spine Deployment and Best Practices with OS10EE. Fabric Design Center (FDC) The Dell EMC Fabric Design Center (FDC) is a cloud-based application that automates the planning, design and deployment of network fabrics that power Dell EMC compute, storage and hyper-converged infrastructure solutions, including VxRail. -
Page 8: Supported Switches And Operating Systems
Supported switches and operating systems The examples provided in this Deployment Guide use VxRail 4.7.0 nodes connected to Dell EMC Networking S5248F-ON switches running Dell EMC Networking Dell EMC OS10 Enterprise Edition (OS10EE) 10.4.2. Dell EMC Networking supports the following switch and OS combinations for VxRail 4.7 and later:... -
Page 9: Hardware Overview
Dell EMC VxRail E560 The Dell EMC VxRail E series consists of nodes that are best suited for remote office or entry workloads. The E series nodes support up to 40 CPU cores, 1536GB memory, and 16TB hybrid or 30TB all-flash storage in a 1-RU form factor. -
Page 10: Dell Emc Networking Z9264F-On
The Dell EMC Networking S5248F-ON is a 1-RU fixed switch with 48x 25 GbE, 4x multirate 100 GbE, and 2x 200 GbE ports, and supports L2 static VXLAN with VLT. The example within this document uses four S5248F-ON switches in VLT pairs, as leaf switches. -
Page 11: Topology
Figure 8 shows the connections and the associated VLANs, with the S5248F-ON switches configured as a VLT pair. As shown in Figure 8, the VxRail node iDRAC connects to the Dell EMC Networking S3048-ON, which serves as the external management switch. - Page 12 Figure 9 shows a physical view of Rack 1. The VxRail P series nodes, sfo01w01vxrail01 through 04 each have 2x 25 GbE links with each being connected to one of the two S5248F-ON leaf switches in the rack. Each VxRail node has an iDRAC connected to a S3048-ON OOB management switch. This connection is used for the initial node configuration.
-
Page 13: Underlay Network Design
OSPF LSA database at a minimum size. Note: For detail instructions on creating a leaf-spine underlay, including considerations and alternative configurations, see Dell EMC Networking Layer 3 Leaf-Spine Deployment and Best Practices with OS10EE. Spine 1 Spine 2... - Page 14 Stack ID sfo01-leaf04 S5248F-ON Stack ID sfo02-leaf03 S5248F-ON Stack ID sfo02-leaf04 Physical switch topology Note: All switch configuration commands are provided in the file attachments. See Section 1.4 for instructions on accessing the attachments. Dell EMC VxRail Multirack Deployment Guide...
-
Page 15: Virtual Extensible Lan (Vxlan) Overlay
Rack 1 Rack 2 VLAN Name External Management Internal Management vSAN vMotion Guest VM Networks Static VXLAN logical diagram Note: For more information about static VXLAN concepts, see the OS10 Enterprise Edition User Guide. Dell EMC VxRail Multirack Deployment Guide... -
Page 16: External Vcenter Connectivity
Spine 2 Z9264-ON Z9264-ON OSPF Area 0 sfo01-leaf01 sfo01-leaf02 sfo01-leaf03 sfo01-leaf04 sfo-edge01 10.222.222.3 10.222.222.1 10.222.222.1 10.222.222.3 10.222.222.64 sfo-edge03 sfo01m01 sfo01w01 Management cluster VxRail Nodes VLAN Name External Management Existing Management Accessing existing data center services Dell EMC VxRail Multirack Deployment Guide... -
Page 17: Switch Configuration Values And Prerequisites
192.168.20.253 Tagged Note: Gateway addresses are provided for each network for future expansion of the environment, including stretched vSAN and vMotion of VMs to other independent clusters. For information about routing L2 VXLANs, Appendix Dell EMC VxRail Multirack Deployment Guide... -
Page 18: Switch Settings
Each switch is in a factory default state 4.2.1 Verify OS10EE version The Dell EMC Networking S5248F-ON switches must have OS10EE version 10.4.2 or later to support Network Virtualization Overlays (NVOs). Run the show version command to check the operating system version. -
Page 19: Verify License Installation
Note: By default, OS10EE has Telnet disabled, SSH enabled, and the OOB management interface that is configured to get an IP address using DHCP. The default username and password are both admin. Dell EMC recommends changing the admin password to a complex password when logging in for the first time. -
Page 20: Configure Switches
3. It is a best practice to restrict Telnet or SSH connections to the switch by applying an access list (ACL) on the Virtual Terminal Line (VTY). The ACL permit-mgmt-access is created and applied to the VTY. In this example, access is limited to hosts on the 100.67.0.0/13 network. Dell EMC VxRail Multirack Deployment Guide... -
Page 21: Configure Ospf Routing And Upstream Network-Facing Ports
10.0.2.5 sfo01-leaf03(config-router-ospf-1)# exit Note: For guidance on router ID IP addresses, see Dell EMC Networking Layer 3 Leaf-Spine Deployment and Best Practices with OS10EE. 2. Configure the upstream connection to the first spine switch, sfo-spine01. Enter the following... -
Page 22: Configure Virtual Link Trunk (Vlt)
VxRail node interfaces. If all upstream connectivity to the spine layer is lost, the VxRail node interfaces are shut down. sfo01-leaf03(config)# uplink-state-group 1 sfo01-leaf03(conf-uplink-state-group-1)# enable sfo01-leaf03(conf-uplink-state-group-1)# downstream ethernet 1/1/1,1/1/3,1/1/5,1/1/7 sfo01-leaf03(conf-uplink-state-group-1)# upstream ethernet1/1/53-1/1/54 sfo01-leaf03(conf-uplink-state-group-1)# exit Dell EMC VxRail Multirack Deployment Guide... -
Page 23: Configure Virtual Extensible Lans (Vxlan)
10.222.222.4 sfo01-leaf03(conf-vn-vxlan-vni)# remote-vtep 10.222.222.64 sfo01-leaf03(conf-vn-vxlan-vni)# exit sfo01-leaf03(conf-vn-1632)# exit 6. Enter the following commands to configure the VXLAN virtual network 1633, vSAN traffic, with a virtual network interface (VNI) and the appropriate remote VTEPs: Dell EMC VxRail Multirack Deployment Guide... - Page 24 10.222.222.4 sfo01-leaf03(conf-vn-vxlan-vni)# remote-vtep 10.222.222.64 sfo01-leaf03(conf-vn-vxlan-vni)# exit sfo01-leaf03(conf-vn-20)# exit Note: The remote VTEP 10.222.222.64 is used to access external sources outside of the VXLAN tunnel. For more information about routing between VXLAN tunnels, see Appendix Dell EMC VxRail Multirack Deployment Guide...
-
Page 25: Configure Virtual Lans (Vlan)
10 sfo01-leaf03(conf-if-vl-10)# no shutdown sfo01-leaf03(conf-if-vl-10)# exit 6. Enter the following commands to configure VLAN 20 for the second guest network traffic, and assign the VLAN to the VXLAN virtual network: Dell EMC VxRail Multirack Deployment Guide... -
Page 26: Configure Downstream Vxrail Node Interfaces
Configure downstream VxRail node interfaces To configure the four downstream interfaces connecting to the Dell EMC VxRail P-Series nodes, perform the following steps: 1. Enter the following commands to configure the downstream interface for the first VxRail node Configure the interface as a trunk link. - Page 27 10,20,1632- 1633,3939 sfo01-leaf03(conf-if-eth1/1/7)# mtu 9216 sfo01-leaf03(conf-if-eth1/1/7)# flowcontrol receive on sfo01-leaf03(conf-if-eth1/1/7)# flowcontrol transmit on sfo01-leaf03(conf-if-eth1/1/7)# spanning-tree port type edge sfo01-leaf03(conf-if-eth1/1/7)# exit Note: See Appendix B for information about MTU considerations. Dell EMC VxRail Multirack Deployment Guide...
-
Page 28: Switch Validation
Switch validation After switches are configured and devices are connected, the Dell EMC OS10 Enterprise Edition (OS10EE) CLI is used to validate the network configuration. The following commands and outputs are for sfo01- leaf03. The output of its peer, sfo01-leaf04, and the remaining S5248F-ON switches are similar. This section provides a list of the most common commands and their output for the examples that are used in this guide. -
Page 29: Show Lldp Neighbors
1/1/3, 1/1/5, and 1/1/7) and are configured for both receive and transmit. sfo01-leaf03# show interface | grep "Flowcontrol rx on tx on" Flowcontrol rx on tx on Flowcontrol rx on tx on Flowcontrol rx on tx on Flowcontrol rx on tx on Dell EMC VxRail Multirack Deployment Guide... -
Page 30: Ospf Validation Commands
10.222.222.0/24 subnet is used for VTEP endpoint IP addresses. Each of the following IP addresses has two paths - one through sfo-spine01, and another through sfo-spine02. The remaining addresses are to the various L3 interfaces that are configured in the leaf-spine underlay. Dell EMC VxRail Multirack Deployment Guide... -
Page 31: Show Ip Ospf Topology
-/-/-/ 192.168.2.10 ethernet1/1/54 0.0.0.0 10.0.2.7 -/-/-/ 192.168.1.10 ethernet1/1/53 0.0.0.0 10.0.2.7 -/-/-/ 192.168.2.10 ethernet1/1/54 0.0.0.0 10.0.2.8 -/-/-/ 192.168.1.10 ethernet1/1/53 0.0.0.0 10.0.2.8 -/-/-/ 192.168.2.10 ethernet1/1/54 0.0.0.0 10.0.2.64 -/-/-/ 192.168.1.10 ethernet1/1/53 0.0.0.0 10.0.2.64 -/-/-/ 192.168.2.10 ethernet1/1/54 0.0.0.0 Dell EMC VxRail Multirack Deployment Guide... -
Page 32: Vxlan Validation Commands
IP Address: 10.222.222.64, State: up, Encap: VxLAN VNI list: 10,20,1631-1633,3939 Note: The remote VTEP 10.222.222.64 is used to access external sources outside of the VXLAN tunnel. For more information routing between L2 VXLAN tunnels, see Appendix Dell EMC VxRail Multirack Deployment Guide... -
Page 33: Show Nve Vxlan-Vni
Codes: DP - MAC-learn Dataplane, CP - MAC-learn Controlplane, UUD - Unknown- Unicast-Drop Virtual Network: 3939 Description: sfo01-w01-vxrail_internal_management Members: VLAN 3939: port-channel1000, ethernet1/1/1, ethernet1/1/3, ethernet1/1/5, ethernet1/1/7 VxLAN Virtual Network Identifier: 3939 Source Interface: loopback0(10.222.222.3) Remote-VTEPs (flood-list): 10.222.222.4(DP) Dell EMC VxRail Multirack Deployment Guide... -
Page 34: Vlt Validation Commands
Note: The Peer HeartBeat status must be Up. sfo01-leaf03# show vlt all backup-link VLT Backup Link ------------------------ Destination : 100.67.167.34 Peer Heartbeat status : Up Heartbeat interval : 30 Heartbeat timeout : 90 Destination VRF : default Dell EMC VxRail Multirack Deployment Guide... -
Page 35: Show Vlt All Mismatch
No mismatch Virtual Network Mode Mismatch: No mismatch Virtual Network Tagged Interfaces Mismatch: No mismatch Virtual Network Untagged Interfaces Mismatch: No mismatch Virtual Network VNI Mismatch: No mismatch Virtual Network Remote-VTEP Mismatch: No mismatch Dell EMC VxRail Multirack Deployment Guide... -
Page 36: Show Vlt Mac-Inconsistency
Note: Verify that the No inconsistencies found status displays. sfo01-leaf03# show vlt mac-inconsistency virtual-network Inconsistency check for Virtual-Network based MAC ------------------------------------------------- Fetching VN MACs from unit 2 Fetching VN MACs from unit 1 Comparing MACs of VLT Peers .. No inconsistencies found Dell EMC VxRail Multirack Deployment Guide... -
Page 37: Perform Initialization To Create Vxrail Cluster
Note: In this document, Rack 1 houses four E-series nodes, where Rack 2 houses four P-series nodes. For ease of manageability, each rack contains a pair of Dell EMC Networking S5248F-ON switches. 2. Attach the appropriate cabling between the ports of the VxRail nodes and the switch ports. -
Page 38: Vxrail Validation
VxRail manager cluster expansion VxRail validation Once the deployment and expansion of the initial cluster are completed, use the VxRail Manager to verify that all eight hosts are in the single cluster. VxRail Manager logical view Dell EMC VxRail Multirack Deployment Guide... -
Page 39: Vmware Vsan Stretched Clusters
Mgmt VLAN: 1731 appliance Witness VLAN: 1734 vSAN stretched cluster network diagram Note: The vSAN witness appliance requires available through routing. The witness appliance can be located in the Primary data center, depending on requirements. Dell EMC VxRail Multirack Deployment Guide... -
Page 40: Deploying Vmware Vsan Stretched Clusters
Figure 19 shows the appliance after it is added to the sfo01-w01dc data center object using the Management Network IP address. The icon for a vSAN witness appliance is blue, which represents that it is an imbedded ESXi appliance and not a physical server. Dell EMC VxRail Multirack Deployment Guide... -
Page 41: Configure Vsan Witness Appliance Routing
64 bytes from 172.16.31.101: icmp_seq=1 ttl=60 time=0.807 ms 64 bytes from 172.16.31.101: icmp_seq=2 ttl=60 time=0.747 ms --- 172.16.31.101 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.641/0.732/0.807 ms Dell EMC VxRail Multirack Deployment Guide... -
Page 42: Configure Network Interfaces For Witness Traffic
Figure 20 shows the vSphere Client and the management VMkernel network interface is now enabled for vSAN Witness traffic but not vSAN traffic. Do no enable the interface for vSAN traffic in the vSphere client. Dell EMC VxRail Multirack Deployment Guide... -
Page 43: Configure Fault Domains
These settings are specific to the vSAN witness appliance. Witness appliance settings Setting Value Local VMware Disk (mpx.vmhba1:C0:T2:L0) cache tier Local VMware Disk (mpx.vmhba1:C0:T1:L0) capacity tier Note: For step-by-step instructions on deploying fault domains, see Manually Configure vSAN Stretched Cluster. Dell EMC VxRail Multirack Deployment Guide... -
Page 44: Witness Component Validation
Preferred fault domain and sfo01w01vxrail06 in the Secondary fault domain, and the witness component is located on sfo03w01vsanw01.The remaining three objects are configured similar with some components being located on alternative hosts in the Secondary fault domain. VxRail Manager virtual object placement Dell EMC VxRail Multirack Deployment Guide... -
Page 45: A Routing In A Layer 2 Vxlan Overlay
VLAN routing. Traffic is then re-encapsulated by sfo-edge01 to reach the target destination Note: Router redundancy is not shown in this example for simplicity. Best practices for router redundancy should be considered during deployment. Dell EMC VxRail Multirack Deployment Guide... - Page 46 Q: A - Access (Untagged), T - Tagged Status Description Q Ports Active sfo01-w01-guest_vm_network_a T Po10 Active sfo01-w01-guest_vm_network_b T Po10 1611 Active sfo01-m01-esxi-management T Po10 1631 Active sfo01-w01-vxrail_external_ma T Po10 1632 Active sfo01-w01-vmotion T Po10 1633 Active sfo01-w01-vsan T Po10 Dell EMC VxRail Multirack Deployment Guide...
-
Page 47: B Maximum Transmission Unit (Mtu) Considerations
9050 or 9054 bytes needs to be allowed to avoid fragmentation. Dell EMC Networking provides a maximum MTU value of 9216 bytes. Jumbo frame MTU of 9000 bytes from the VxRail node can be accommodated by the network switches without introducing fragmentation. The jumbo frame MTU needs to be configured consistently across all points in the network, including the ESXi virtual switches. -
Page 48: C Vxrail Deployment Values
Subnet mask 255.255.255.0 VLAN ID 1632 vSAN Starting address for IP pool 172.16.33.101 Ending address for IP pool 172.16.33.165 Subnet mask 255.255.255.0 VLAN ID 1633 VM Networks Guest_VM_Network_A Guest_VM_Network_B Solutions Logging vRealize Log Insight hostname Dell EMC VxRail Multirack Deployment Guide... -
Page 49: D Connecting A Workstation Or Laptop For Vxrail Initialization
Eth 100g-2x port-group1/1/14 Eth 100g-2x port-group1/1/15 Eth 100g-1x port-group1/1/16 Eth 100g-1x port-group1/1/17 Eth 100g-1x port-group1/1/18 Eth 100g-1x 3. Use the port-group command mode to set all physical interfaces in the port group to 10 GbE. Dell EMC VxRail Multirack Deployment Guide... - Page 50 Duplex Mode Vlan Tagged-Vlans Eth 1/1/1 sfo01w01.. full 1631 10,20,1632-1633,3939 Eth 1/1/3 sfo01w01.. full 1631 10,20,1632-1633,3939 Eth 1/1/5 sfo01w01.. full 1631 10,20,1632-1633,3939 Eth 1/1/7 sfo01w01.. full 1631 10,20,1632-1633,3939 Eth 1/1/32:1 1000M full 1631 – Dell EMC VxRail Multirack Deployment Guide...
-
Page 51: Validated Components
Dell EMC Networking S5248F-ON leaf switches 10.4.2.236 Dell EMC Networking Z9264F-ON spine switches 10.4.2.236 Dell EMC Networking S3048-ON OOB mgmt switches 10.4.2.236 VxRail E560 nodes A cluster of four VxRail E560 nodes was used to validate the examples in this guide. The nodes were each configured using the information that is provided below. - Page 52 Item Version Dell PTAgent 1.8.1-21 VMware ESXi 6.7.0 build 10764712 VxRail Manager 4.7.001 11073036 VMware vCenter Server Appliance 6.7.0 build 10244857 VxRail Manager VIB 4.7.001-11073036 VMware vSAN vSAN 6.7 Update 1, provided with ESXi Dell EMC VxRail Multirack Deployment Guide...
- Page 53 Technical resources Dell EMC Networking Guides OS10 Enterprise Edition Users Guide 10.4.2.0 Edition Dell EMC Networking Layer 3 Leaf-Spine Deployment and Best Practices with OS10EE Dell EMC VxRail Network Guide VxRail Appliance 4.7 Administration Guide VxRail Planning Guide for Virtual SAN Stretched Cluster VMware Validated Design for Software-Defined Data Center 4.3 Release Notes...
- Page 54 Support and feedback Contacting Technical Support Support Contact Information Web: http://www.dell.com/support Telephone: USA: 1-800-945-3355 Feedback for this document To provide feedback on the quality and usefulness of this publication, send an e-mail to Dell_Networking_Solutions@Dell.com. Dell EMC VxRail Multirack Deployment Guide...
Need help?
Do you have a question about the VxRail P series and is the answer not in the manual?
Questions and answers