Preface • Overview..................................5 • Purpose of This Document............................6 • Audience..................................6 • Objectives..................................7 • Document History................................. 7 Overview Ruckus Campus Fabric technology integrates high-performance, fixed-form-factor switches to create a single distributed logical switch that is independent of physical location and that allows organizations to add ports across the campus whenever and wherever needed without adding complexity.
Preface Purpose of This Document The Ruckus ICX 7250 and ICX 7450 switches deliver the performance, flexibility, and scalability required for enterprise gigabit Ethernet access deployment. These switches offer market chassis-level performance and reliability with the flexibility, cost- effectiveness, and "pay as you grow" scalability of a stackable solution. The components of the Ruckus SPX architecture are the Ruckus ICX 7250 and ICX 7450 routers configured as port extenders (PEs), or PE units, to a set of Ruckus ICX 7750 stack units configured as the 802.1BR control bridge.
Preface Document History Objectives The objective of this guide is to assist the administrator in bringing up and configuring the Switch Port Extender. This deployment guide covers the following topics in depth: • SPX technical architecture • SPX topologies • SPX construction methods •...
Page 8
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
SPX Technical Architecture • Control Bridge................................9 • Port Extenders................................9 Control Bridge The control bridge (CB) manages port extender (PE) units. It performs switching, routing, and forwarding for PE ports and provides centralized policy management. The CB uses the Link Layer Discovery Protocol (LLDP) to discover PE units. When the CB discovers a PE unit, it connects to the PE unit and creates a control plane using Control and Status Protocol (CSP) over the uplink/ cascade port.
Page 10
SPX Technical Architecture Port Extenders NOTE The Ruckus ICX 7450-ZP does not support 802.1BR in FastIron 08.0.40. The ICX 7250 has the following models: • ICX7250-24 • ICX7250-24P • ICX7250-48 • ICX7250-48P NOTE The Ruckus ICX 7250-24G does not support 802.1BR in FastIron 08.0.50. The ICX 7150 has the following models: •...
Supported SPX Topologies FIGURE 2 Topology 1: SPX-LAG Spanned Across the CB Stack Unit FIGURE 3 Topology 2: Multiple PE Chains Connected to the Same CB Unit Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Page 12
Supported SPX Topologies FIGURE 4 Topology 3: SPX-LAG Between PE Units FIGURE 5 Topology 4: SPX with Standalone CB Unit Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Page 13
Supported SPX Topologies FIGURE 6 Topology 5: SPX PE Ring Topology from PE to CB Units The green X indicates the location of the logical block for data flow through the PE ring. The logical block is placed on the link that comes up last during the PE ring formation.
Page 14
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Unsupported SPX Topologies FIGURE 7 Topology 1: Ring Topology Within a PE Chain FIGURE 8 Topology 2: Ring Topology from PE to CB Units Is Not Supported in 08.0.40 Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Page 16
Unsupported SPX Topologies FIGURE 9 Topology 3: SPX-LAG Spanning Multiple PE Chains FIGURE 10 Topology 4: More Than One PE Unit Connected to a Transit PE Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Page 17
Unsupported SPX Topologies FIGURE 11 Topology 5: Edge PE Connected to an Existing PE Ring FIGURE 12 Topology 6: SPX PE Ring with One PE Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Page 18
Unsupported SPX Topologies FIGURE 13 Topology 7: PE Chains of Seven or More PEs Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Scaling (Supported Topologies) FIGURE 14 Scaled Topology Including SPX (PE) Ring and PE Chains Scalability Limits • 4 CB stack units (ICX 7750) • 36 PE units (ICX 7450, ICX 7250, and ICX 7150) • 6 PE units per chain cascade depth •...
Page 20
Scaling (Supported Topologies) ATTENTION Use port extender ports to hook to APs, phones, and PoE devices. Use control bridge ports to connect to servers, VMs, and so on. Do not hook the PE ports to any other routers and switches. Do not deploy where compact switches are connected to the PE ports where the compact switches are not part of the Campus Fabric, but standalone units.
SPX Construction Methods • Prerequisites................................21 • Valid Port Combinations............................. 21 Prerequisites • The control bridge can be a standalone unit or a stack (ring or linear). • Port extenders can be cascaded up to a depth of six. A ring topology of a PE chain is supported beginning with FastIron 08.0.50 and later.
Page 22
SPX Construction Methods Valid Port Combinations TABLE 2 PE-to-PE Port Combinations (continued) PE1 Port PE2 Port 1G Fiber — — — — — (Base Unit Ports) 2.5G Copper — — — — (Base Unit Ports): ICX 7450-32ZP NOTE For zero-touch provisioning (ZTP), it is mandatory to use nonbase module ports on ICX 7450, ICX 7250, or ICX 7150 units. Nonbase modules are the modules other than module 1.
Bill of Materials The following products are used in this deployment. TABLE 3 Products Used in ICX 7750 CB Stacking Identifier Vendor Model Notes Switch/Units Ruckus ICX 7750-48C The CB stack can be formed with a mix of all three different models listed.
Page 24
Bill of Materials TABLE 5 Products Used in ICX 7250 PE Ports (continued) Identifier Vendor Model Notes License Needed for 10- Ruckus Yes (2x10 or 8x10 PoD license) A license is not needed for 1-Gbps SPX port. GbpsSPX port/SPX LAG TABLE 6 Products Used in ICX 7150 PE Ports (FastIron 08.0.70 or Later) Identifier Vendor...
Page 25
Bill of Materials TABLE 7 Optics Information for ICX 7750 Stacking (continued) QSFP Hardware Description Ruckus Part Number 40G-QSFP-4SFP-C-0101 4x10GE QSFP+ to 4 SFP+copper cable: 1 m 58-0000051-01 40G-QSFP-4SFP-C-0301 4x10GE QSFP+ to 4 SFP+copper cable: 3 m 58-0000052-01 40G-QSFP-4SFP-C-050 4x10GE QSFP+ to 4 SFP+copper cable: 5 m 58-0000053-01 10GE SR SFP+ 10G-SFPP-SR...
Page 26
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Deployment Considerations • SPX LAG or port: Only one SPX LAG or SPX port is allowed between the CB and the PE or between the PE and the PE. • Speed: No speed negotiation is possible when SPX links are configured. •...
Page 28
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Bringing Up SPX • Configuration Notes..............................29 • Reference Topology..............................30 • Configuring the CB..............................30 • Configuring the PE...............................32 Configuration Notes • An ICX 7750 traditional stack or a standalone ICX 7750 device can be enabled as an 802.1BR control bridge (CB). •...
Bringing Up SPX Reference Topology Reference Topology FIGURE 15 Bringing Up SPX Reference Topology Configuring the CB Enable SPX on the ICX 7750. • Configure spx cb-enable on the active CB unit. The spx cb-enable command adds "lldp run" to the running configuration in FastIron 08.0.40 and earlier.
Page 31
Bringing Up SPX Configuring the CB Configure the PE IDs and name. • Optionally, you can configure the PE ID using the pe-id command. • If you do not configure the PE ID, the system assigns the available PE IDs based on the join time of the PEs. PE Chain 17, 18 ICX7750-48F Router(config)# spx cb-configure ICX7750-48F Router(config-spx-cb)# pe-id 1/1/1 17 18...
Bringing Up SPX Configuring the PE Configuring the PE Enable SPX on the ICX 7450 or ICX 7250. • On a standalone ICX 7450, use the spx pe-enable command to bring up the unit in provisional PE mode. When the unit is PE-enabled, the system generates two default SPX ports for the PE.
Page 33
Bringing Up SPX Configuring the PE PE-19 ICX7450-24 Router(config)# spx pe-enable Enter provisional PE mode. CLI is limited to spx unit 1. After finishing all configuration, please "write memory" and reload this unit to be a PE. [Provisional-PE]ICX7450-24 Router(config)# show run Current configuration: ver 08.0.40bT213 spx pe-enable...
Page 34
Bringing Up SPX Configuring the PE Configure the SPX port or LAG on the PE. • Use the spx-port or spx-lag configuration command for the PE-CB link (in this case, PE 17 and 19). • Use the spx-port or spx-lag configuration command for the PE-PE link (in this case, PE 18). PE 17 PE 17 1/2/1 and 1/2/2 are connected to CB unit 1, and 1/2/3 is connected to PE unit 18.
Page 35
Bringing Up SPX Configuring the PE Save the configuration and reload. • Once a PE is enabled and the appropriate SPX port or LAG is configured, you must use the write memory and reload commands to complete initialization. • The ICX 7450 or ICX 7250 enters PE mode with the assigned PE ID from the CB. •...
Page 36
Bringing Up SPX Configuring the PE From FastIron 08.0.70 and later, all models of the ICX 7150 can be added as a port extender. ICX7750-48F Router# show spx T=22d5h16m24.5: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address Pri State Comment S ICX7750-48XGF active...
SPX PE Ring • Overview..................................37 • How a PE Ring Works with Different Types of Traffic....................38 Overview SPX (802.1BR) was introduced to the ICX system in FastIron 08.0.40. SPX includes a CB and a chain of PEs attached to the CB (for example, 1 CB with 4 PEs attached to it in a chain).
SPX PE Ring How a PE Ring Works with Different Types of Traffic How a PE Ring Works with Different Types of Traffic Unicast Traffic FIGURE 17 PE Ring in Unicast Traffic Stream 1 [S1] • Ingress traffic from VM3: Takes the shortest upstream path to the CB, SPX Link B. •...
SPX PE Ring How a PE Ring Works with Different Types of Traffic Multicast/Broadcast and Unknown Unicast Traffic FIGURE 18 PE Ring in Multicast/Broadcast and Unknown Unicast Traffic Stream 1 [S1] • Ingress traffic from VM3: Takes the shortest upstream path to the CB, SPX Link B. •...
Page 40
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Deployment Scenarios • Reference Topology..............................41 • Deployment Scenario 1: Deploying a PE Ring in an Existing Campus Fabric Domain........42 • Deployment Scenario 2: Deploying a PE Ring Using ZTP..................43 • Deployment Scenario 3: Adding a New PE Unit to an Existing PE Ring..............44 •...
Deployment Scenarios Deployment Scenario 1: Deploying a PE Ring in an Existing Campus Fabric Domain Deployment Scenario 1: Deploying a PE Ring in an Existing Campus Fabric Domain Follow the instructions in SPX Construction Methods on page 21 to create a Campus Fabric domain with CB and PE chains. The following steps explain how to form a PE ring after a Campus Fabric domain with PE chains comes up.
Deployment Scenarios Deployment Scenario 2: Deploying a PE Ring Using ZTP A syslog message confirms that the PE ring has been deployed. You can also use SPX (PE) ring-related show commands to verify the status of the PE ring as described in Verification Commands: SPX PE Ring on page 72.
Deployment Scenarios Deployment Scenario 3: Adding a New PE Unit to an Existing PE Ring Deployment Scenario 3: Adding a New PE Unit to an Existing PE Ring Consider the following PE ring with five PEs. ICX7750-48F Router# show spx T=22d5h50m43.9: alone: standalone, D: dynamic cfg, S: static Type Role...
Page 45
Deployment Scenarios Deployment Scenario 3: Adding a New PE Unit to an Existing PE Ring The show spx ring all command indicates that the PE ring is in the break state, and the show spx command displays the broken PE ring. ICX7750-48F Router# show spx T=22d5h54m16.0: alone: standalone, D: dynamic cfg, S: static Type...
Page 46
Deployment Scenarios Deployment Scenario 3: Adding a New PE Unit to an Existing PE Ring Bring up the new unit as a PE. ICX7250-48 Router> enable No password has been assigned yet... ICX7250-48 Router# configure terminal ICX7250-48 Router(config)# spx pe-enable Enter provisional PE mode.
Page 47
Deployment Scenarios Deployment Scenario 3: Adding a New PE Unit to an Existing PE Ring Save the configuration, and reload the PE. [Provisional-PE]ICX7250-48 Router(config-if-spx-e10000-1/2/1)# end [Provisional-PE]ICX7250-48 Router# write memory Flash Memory Write (133 bytes) [Provisional-PE]ICX7250-48 Router# Write PE startup file done. [Provisional-PE]ICX7250-48 Router# reload Are you sure? (enter 'y' or 'n'): y After the new PE reboots, the SPX system adds the PE as part of the new six-PE ring.
Deployment Scenarios Deployment Scenario 4: Replacing a PE in an Existing PE Ring Deployment Scenario 4: Replacing a PE in an Existing PE Ring Follow this procedure when a PE in an existing PE ring must be replaced. In this example, PE 22 is replaced with a new PE unit. Suppose that you are working with the following PE ring topology.
Page 49
Deployment Scenarios Deployment Scenario 4: Replacing a PE in an Existing PE Ring Bring up the new replacement unit as a PE. ICX7250-48 Router> enable No password has been assigned yet... ICX7250-48 Router# configure terminal ICX7250-48 Router(config)# spx pe-enable Enter provisional PE mode. CLI is limited to spx unit 1. After finishing all configuration, please "write memory"...
Page 50
Deployment Scenarios Deployment Scenario 4: Replacing a PE in an Existing PE Ring Save the configuration, and reload the PE 22 replacement. [Provisional-PE]ICX7250-48 Router(config-if-spx-e10000-1/2/1)# end [Provisional-PE]ICX7250-48 Router# write memory Flash Memory Write (133 bytes) [Provisional-PE]ICX7250-48 Router# Write PE startup file done. [Provisional-PE]ICX7250-48 Router# reload Are you sure? (enter 'y' or 'n'): y After the reload, the PE replacement unit becomes part of the SPX domain.
Deployment Scenarios Deployment Scenario 5: Moving SPX Links Between Modules in a PE Ring Deployment Scenario 5: Moving SPX Links Between Modules in a PE Ring Follow this procedure to upgrade an existing 1-Gbps SPX link between PEs to a 10-Gbps link. Use the same procedure to downgrade the 10-Gbps SPX link to 1 Gbps.
Page 52
Deployment Scenarios Deployment Scenario 5: Moving SPX Links Between Modules in a PE Ring The show spx ring all command confirms that the PE ring is in the break state. The show spx command shows two PE chains. ICX7750-48F Router# show spx ring all ------------------------------------------------------------------------------------ Ring Id |FSM State |CB port |Remote CB port...
Page 53
Deployment Scenarios Deployment Scenario 5: Moving SPX Links Between Modules in a PE Ring Ports 18/1/13 and 19/1/13 no longer appear in the show spx command output. ICX7750-48F Router(config-spx-unit-19)# show spx T=15d5h21m39.8: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address Pri State...
Deployment Scenarios Deployment Scenario 6: Adding More Links to a Live SPX Link Using the multi-spx-lag Command 12. Once the PE ring is formed, the relevant PE ID configuration is automatically generated in the running configuration. The new PE ID configuration merges the two PE ID chains into one PE ID ring configuration. ICX7750-48F Router# show running-config | include pe-id pe-id 1/1/5 17 18 19 20 21 22 2/1/9 13.
Page 55
Deployment Scenarios Deployment Scenario 6: Adding More Links to a Live SPX Link Using the multi-spx-lag Command 22 D ICX7250-48 spx-pe cc4e.24b4.4350 N/A remote Ready active standby +---+ +---+ +---+ -3/1| 1 |3/4--3/1| 3 |3/4--3/1| 2 |3/4- +---+ +---+ +---+ |-------------------------------------| +----+ +----+...
Deployment Scenarios Deployment Scenario 7: Removing Links from a Live SPX LAG and Converting the LAG to an SPX Port Enter the multi-spx-lag command to convert the SPX port to an SPX LAG. ICX7750-48F Router# configure terminal ICX7750-48F Router(config)# spx unit 17 ICX7750-48F Router(config-spx-unit-17)# multi-spx-lag 17/3/1 to 17/3/3 and 18/3/1 to 18/3/3 Warning! multi-spx-lag may cause some PEs to be unreachable.
Page 57
Deployment Scenarios Deployment Scenario 7: Removing Links from a Live SPX LAG and Converting the LAG to an SPX Port Type Role Mac Address Pri State Comment S ICX7750-48XGF active cc4e.24d0.3500 128 local Ready S ICX7750-48XGC standby 609c.9f1f.df00 128 remote Ready S ICX7750-48XGF member cc4e.246e.d580...
Page 58
Deployment Scenarios Deployment Scenario 7: Removing Links from a Live SPX LAG and Converting the LAG to an SPX Port Use the multi-spx-port command to convert the SPX LAG to an SPX port. ICX7750-48F Router(config)# spx unit 17 ICX7750-48F Router(config-spx-unit-17)# multi-spx-port 17/3/1 and 18/3/1 Warning! multi-spx-port may cause some PEs to be unreachable.
Deployment Scenarios Deployment Scenario 8: Changing a PE Ring to a Linear PE Chain Deployment Scenario 8: Changing a PE Ring to a Linear PE Chain Follow these steps to remove a PE ring and convert it to two PE chains. Reference Topology on page 41 includes a PE ring.
Page 60
Deployment Scenarios Deployment Scenario 8: Changing a PE Ring to a Linear PE Chain Enter the show spx ring all command to confirm that the PE ring is in the break state. Enter the show spx command to display the two PE chains. ICX7750-48F Router# show spx ring all ------------------------------------------------------------------------------------ Ring Id |FSM State |CB port...
Page 61
Deployment Scenarios Deployment Scenario 8: Changing a PE Ring to a Linear PE Chain Enter the show spx command to confirm that SPX ports 20/1/1 and 21/1/1 are no longer present. ICX7750-48F Router# show spx T=14d8h44m0.8: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address...
Page 62
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Image Upgrade • An 802.1BR system requires that all CB and PE units run the same image versions. • An image copy requires two steps in an SPX system: Copy the image to the CB stack units. Copy the image to the PE units. Both steps can be completed through a CB management session.
Page 64
Image Upgrade | 1 |2/1==2/1| 2 |2/3==2/1| 3 |2/5==2/5| 4 |2/1==2/5| 5 |2/1= +----+ +----+ +----+ +----+ +----+ +----+ | 6 |2/1= +----+ 2/1/9 Discovered 1 chain/ring Chain #0: Do you want to select this chain? (enter 'y' or 'n'): y #1: icx7450-48p-poe 609C.9F2A.D460, *** image mismatch, type an ID (No: 0, default: 17): #2: icx7450-48p-poe 609C.9F2A.97E0, type an ID (No: 0, default: 18): #3: icx7250-24-port 609C.9F04.6F90, *** image mismatch, type an ID (No: 0, default: 21):...
Page 65
Image Upgrade If a TFTP server is not configured, the following errors show up for SPX interactive-setup or ZTP. Zero-touch discovers 1 chain (valid#= 1) ZTP detects 3 units running different images, but "tftp-server" is not configured. Abort SYSLOG: <14> Mar 24 19:59:14 ZTP: zero-touch-enable detects total 1 chains (6 units). unstable= spx interactive-setup discovers 1 chain (valid#= 1, selected#= 1) Detects 3 units running different images but "tftp-server"...
Page 66
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Debugging and Verification Commands • Verification Commands: SPX PE Ring........................72 • Debugging PE Ring Failures............................74 Once the SPX domain is formed, you can use show commands to verify that the SPX is operational. FIGURE 20 show spx Command • The show spx command on the CB displays the CB stack and PE unit/PE chain with a cascade port.
Page 68
Debugging and Verification Commands FIGURE 21 show spx csp all Command • The show spx csp all command can be used to indicate how far the CSP went forward. • CSP Oper status "Yes" shows that CSP is operational and that the PE is attached and operational. •...
Page 69
Debugging and Verification Commands FIGURE 22 show spx csp unit-id Command The show spx csp unit-id command can be used to see whether VP port creation succeeded and whether the ECID is assigned to all PE ports. Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Page 70
Debugging and Verification Commands Example show Commands on the PE [PE]local-17@ICX7450-48P Router# show running-config Current configuration: ver 08.0.40bT213 spx pe-enable spx unit 17 module 1 icx7450-48p-poe-management-module module 2 icx7400-xgf-4port-40g-module module 3 icx7400-qsfp-1port-40g-module module 4 icx7400-qsfp-1port-40g-module spx-lag 17/2/1 to 17/2/2 spx-port 17/2/3 [PE]local-17@ICX7450-48P Router# show spx csp all CSP Oper: yes, Attach time: 1m5.1, up time: 16 hour(s) 31 minute(s) 27 second(s) PE Spx Lag id: 1, cur state up, IPC/ECP Port: 17/2/1...
Page 71
Debugging and Verification Commands spx unit 19 module 1 icx7450-24-port-management-module module 2 icx7400-qsfp-1port-40g-module module 3 icx7400-qsfp-1port-40g-module module 4 icx7400-qsfp-1port-40g-module spx-lag 19/2/1 19/3/1 19/4/1 [PE]local-19@ICX7450-24 Router# show spx csp all CSP Oper: yes, Attach time: 30.1, up time: 16 hour(s) 49 minute(s) 9 second(s) PE Spx Lag id: 1, cur state up, IPC/ECP Port: 19/4/1 Number of Traffic Class: 8 Priority Flow Control: no...
Debugging and Verification Commands Verification Commands: SPX PE Ring active standby +---+ +---+ +---+ -2/1| 1 |2/4--2/1| 3 |2/4--2/4| 2 |2/1- +---+ +---+ +---+ |-------------------------------------| Standby u2 - protocols ready, can failover Current stack management MAC is cc4e.246e.3480 Image-Auto-Copy is Enabled. Stack Port Status Neighbors Unit# Stack-port1...
Page 73
Debugging and Verification Commands Verification Commands: SPX PE Ring SPX CB [CASC PORT] CHAINS CB Port | Epoch |Ring FSM State|Remote CB port Chain ---------------------------------------------------------------------------------------------- 2/1/9 3075 BREAK 1/1/5 3079 PE22--PE21 1/1/5 3079 BREAK 2/1/9 3075 PE17--PE18--PE19--PE20 show spx ring ring-ID The output lists all PE units in the ring (under "Chain"), identifies the upstream ("Uplink") and downstream ("Cascade") port for each PE unit in the ring, specifies the previous and next PE in the chain, identifies the logical block point, and displays the state of each PE.
Debugging and Verification Commands Debugging PE Ring Failures Debugging PE Ring Failures SPX (PE) Ring Is Not in the Active State Enter the show spx ring all command to confirm that the PE ring is not in the Active state. In the following example, the PE ring is in the break state.
Page 75
Debugging and Verification Commands Debugging PE Ring Failures Port 17/3/1 flaps. SYSLOG: <14> Nov 9 15:53:00 System: Interface ethernet 17/3/1, state down SYSLOG: <13> Nov 9 15:53:01 SPX Ring Instance 1 state change to BREAK (CB2/1/9 - CB1/1/5). Logical block: Ring is broken - No Logical Block SYSLOG: <14>...
Page 76
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
PE Ring FSM States There are four Finite State Machine (FSM) states for a PE ring: • INIT: The PE chains are in the Init state when there is no PE ring in the setup. ICX7750-48F Router# show spx ring all ------------------------------------------------------------------------------------ Ring Id |FSM State |CB port |Remote CB port...
Page 78
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Best Practices One recommended method for creating a PE ring is to connect two edge PEs in an existing SPX domain, as shown in the following figure. The procedure for forming an PE ring is explained in Deployment Scenario 1: Deploying a PE Ring in an Existing Campus Fabric Domain on page 42.
Page 82
Best Practices FIGURE 24 Connecting the Edge PE of the Existing PE Chain to the CB • Beginning with FastIron 08.0.70, SPX interactive-setup option 2&3 can detect new links between a CB unit and an edge PE or between two edge PE units. When these links are detected, the associated PE chain or, if present, both associated PE chains are converted into a ring.
Page 83
Best Practices ICX7750-48F Router(config)# show run | beg spx unit spx unit 17 pe-name workstations module 1 icx7450-48p-poe-management-module module 2 icx7400-xgf-4port-40g-module module 3 icx7400-qsfp-1port-40g-module module 4 icx7400-qsfp-1port-40g-module spx-lag 17/2/1 to 17/2/2 spx-port 17/2/3 • If a PE in jumbo mode does not match the CB mode (for example, if a PE is jumbo-disabled and the CB is jumbo- enabled), the PE will reload soon after joining the CB.
Page 84
Ruckus ICX Switch Port Extender Deployment Guide Part Number: 53-1004186-03...
Zero-Touch Provisioning PE Candidate Requirements FIGURE 26 ZTP on a CB Stack in a New or Existing Campus Fabric (SPX) Domain FIGURE 27 ZTP on an Existing SPX Domain PE Candidate Requirements • The new unit must be a clean standalone unit without any startup configuration. •...
Zero-Touch Provisioning Topology Requirements Topology Requirements • New units can be added to a CB (an ICX 7750 standalone or stack) or to an existing Campus Fabric domain. • PE units can be added in a linear or ring topology up to a cascade depth of six units. Refer to Supported Topologies for Zero-Touch on page 88 and...
Zero-Touch Provisioning Supported Topologies for Zero-Touch Supported Topologies for Zero-Touch FIGURE 28 Supported Zero-Touch Topologies Topology 1: New units up to a cascade depth of six can be added in a linear PE topology. Topology 2: New units up to a cascade depth of six can be added in a ring PE topology. Topology 3: New units can be added to the existing SPX domain in a linear PE topology.
Zero-Touch Provisioning Unsupported Topologies for Zero-Touch Unsupported Topologies for Zero-Touch FIGURE 29 Unsupported Zero-Touch Topologies Unsupported Topology 1: A set of new units cannot be discovered if there is a loop between PE units. Unsupported Topology 2: A set of new units cannot be discovered if there is an improper loop between PEs and the CB. Unsupported Topology 3: More than one unit cannot be added directly to an edge PE.
Zero-Touch Provisioning Zero-Touch Features Zero-Touch Features Zero-Touch Probe Ports When zero-touch is enabled on the CB, probe packets are sent from eligible ports to check the topology for potential PE candidates. Any of the following ports may be SPX or ZTP probe ports for discovering new units: •...
Page 91
Zero-Touch Provisioning Zero-Touch Features • All nonbase module ports of an edge PE (all ports from modules 2, 3, and 4 of an edge PE). SPX or ZTP probes for new units cannot be sent over the following ports, and attached units will not be discovered: •...
Page 92
Zero-Touch Provisioning Zero-Touch Features NOTE If a previously configured SPX LAG differs from the ZTP-detected LAG and overlaps with it, the Auto-LAG utility deletes the previous LAG and creates a new LAG. FIGURE 32 Auto-LAG with Zero-Touch In this figure, 2/1/5 to 2/1/8 is represented as one link. Modifying a Live Link with Zero-Touch Zero-touch does not support the following functions: •...
Zero-Touch Provisioning SPX Interactive-Setup NOTE Refer to Deployment Scenario 6: Adding More Links to a Live SPX Link Using the multi-spx-lag Command on page 54 and Deployment Scenario 7: Removing Links from a Live SPX LAG and Converting the LAG to an SPX Port on page 56 for information on using the multi-spx-port and multi-spx-lag commands for making changes to a live SPX link.
Zero-Touch Provisioning Managing PEs Managing PEs Once the Campus Fabric domain is deployed, you may want to optimize PEs in various ways. Zero-touch deployment provides two options for managing PE IDs. Changing PE IDs Previously, changing PE IDs required powering down the units and changing the PE ID configuration. With option 1 of SPX interactive-setup, you can change PE IDs without shutting down the PEs or changing any connections or topology.
Zero-Touch Provisioning Zero-Touch Deployment Considerations FIGURE 35 Moving PEs Move PEs by changing connections or topology but retaining the old PE ID: This is achieved with the spx allow-pe-movement command. Refer to Deployment Scenario 7: SPX PE Movement: Moving PEs Without Changing IDs on page 126.
Zero-Touch Provisioning Deployment Scenarios Deployment Scenarios Deployment Scenario 1: Bringing Up a Complete Campus Fabric Domain with Zero-Touch Initial Deployment Procedure Zero-Touch Provisioning • The user must cable the connections per the topology and PE requirements. • Refer to the Ruckus ICX 7450, ICX 7750, and ICX 7250 Stacking Deployment Guide for information on bringing up a CB (ICX 7750) stack.
Page 97
Zero-Touch Provisioning Deployment Scenarios Enable zero-touch under the CB configuration. SPX# configure terminal SPX(config)# spx cb-configure SPX(config-spx-cb)# zero-touch-enable Configure zero-touch ports or SPX ports or SPX LAGs that connect to the CB. For a linear topology: Specify the zero-touch ports connecting from the CB to the transit PE. For a ring topology: Specify the zero-touch ports connecting from the CB to a PE on both ends.
Page 98
Zero-Touch Provisioning Deployment Scenarios All the new units are set to ready. T=29m11.4: create spx u17, mac=609c.9f39.daa8 PE-port=17/4/1 CB-port=2/2/6 Debug: Oct 26 15:37:49 T=29m12.3: U1 sets spx u17 to ready. Debug: Oct 26 15:37:49 T=29m12.6: create spx u18, mac=cc4e.248b.ad08 PE-port=18/2/4 CB-port=2/2/6 cascade-PE-port=17/2/4 Debug: Oct 26 15:37:50 T=29m13.3: U1 sets spx u18 to ready.
Page 99
Zero-Touch Provisioning Deployment Scenarios Save the configuration. SPX# write memory Send "write mem" to 8 PEs: 17-24 Automatic copy to member units: 2 3 ICX7750-48F Router# Flash Memory Write (8192 bytes per dot) Disable the zero-touch configuration. SPX# configure terminal SPX(config)# spx cb-configure SPX(config-spx-cb)# no zero-touch-enable When all PEs are ready, review the show running-config output to verify the SPX LAG configuration.
Page 100
Zero-Touch Provisioning Deployment Scenarios System Deployment Procedure Using SPX Interactive-Setup FIGURE 37 Reference Topology: Deployment Using SPX Interactive-Setup • The user must cable the connections per the topology and PE requirements. Make sure that the connected PEs do not have a startup configuration. •...
Page 101
Zero-Touch Provisioning Deployment Scenarios Configure zero-touch ports or SPX ports and SPX LAGs that connect to the CB. For a linear topology: Specify the zero-touch ports that connect the CB to the transit PE. For a ring topology: Specify the zero-touch ports that connect the CB to a PE on both ends. SPX# configure terminal SPX(config)# spx cb-configure SPX (config-spx-cb)# zero-touch-ports 1/1/47 2/1/46 2/1/48 2/2/6 3/2/6...
Page 102
Zero-Touch Provisioning Deployment Scenarios Select the chains to be kept in the configuration. The system displays default IDs. Before the system is confirmed, the system prompts you to select or change the default IDs. chain #0: Do you want to select this chain?(enter 'y' or 'n'): y #1: icx7450-24-port-management 609C.9F39.DAA8, type an ID (No: 0, default: 17): 17 #2: icx7450-24-port-management CC4E.248B.AD08, type an ID (No: 0, default: 18): 18 #3: icx7250-24-port-management CC4E.24DF.0E3E, type an ID (No: 0, default: 19): 19...
Zero-Touch Provisioning Deployment Scenarios Verify the PE operational state. SPX# show spx T=20h16m21.5: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address Pri State Comment S ICX7750-48XGF active cc4e.246e.db80 128 local Ready S ICX7750-48XGC standby 609c.9f1f.d900 128 remote Ready S ICX7750-48XGC member 609c.9f1f.d180...
Page 105
Zero-Touch Provisioning Deployment Scenarios • You must cable the connections to the new units per topology and PE requirements. On an ICX 7750 stack or standalone unit, enter the spx cb-enable command. SPX# configure terminal SPX(config)# spx cb-enable If zero-touch is enabled, make sure that the utility is turned off before running SPX interactive-setup. SPX# configure terminal SPX(config)# spx cb-configure SPX(config-spx-cb)# no zero-touch-enable...
Page 106
Zero-Touch Provisioning Deployment Scenarios Run SPX interactive-setup on the CB master. Use option 3 because both new units have a startup configuration. The discovered topology is presented. ICX7750-48F Router# spx interactive-setup You can abort spx interactive-setup at any stage by <ctrl-c> 0: quit 1: change PE IDs 2: discover and convert new units (no startup-config flash) to PEs...
Page 107
Zero-Touch Provisioning Deployment Scenarios Accept the discovered topology when prompted. When prompted, accept the default PE ID for new units, or enter the desired PE ID. chain #0: Do you want to select this chain?(enter 'y' or 'n'): y #1: icx7450-32zp-poe-management 609C.9F1D.AF60, type an ID (No: 0, default: 24): You selected 1 unit(s): #1: ID=24, +----+ +----+...
Zero-Touch Provisioning Deployment Scenarios Verify the PE operational state. SPX# show spx T=20h16m21.5: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address Pri State Comment S ICX7750-48XGF active cc4e.246e.db80 128 local Ready S ICX7750-48XGC standby 609c.9f1f.d900 128 remote Ready S ICX7750-48XGC member 609c.9f1f.d180...
Page 109
Zero-Touch Provisioning Deployment Scenarios Migrating a Linear Stack to the Campus Fabric Domain FIGURE 39 Migrating from a Linear Stack to Campus Fabric From Top Use Case 1: Migrating an existing linear stack to PEs in a linear topology Use Case 2: Migrating an existing linear stack to PEs in a ring topology Procedure Summary The CB and the existing stack must be properly cabled.
Page 110
Zero-Touch Provisioning Deployment Scenarios NOTE Zero-touch or SPX interactive-setup works only if you erase the startup configuration on the existing stack units before attempting to convert them to PE units. Illustration Topology A two-unit ICX 7450 linear stack is cabled to the CB, and the stack units need to be converted to PE units. •...
Page 111
Zero-Touch Provisioning Deployment Scenarios Migrating a Ring Stack to a Campus Fabric Domain FIGURE 40 Migrating from a Ring Stack to Campus Fabric From Top Use Case 1: Converting an existing ring stack to PEs in a linear topology Use Case 2: Converting an existing ring stack to PEs in a ring topology Procedure Summary Connect the CB and the existing stack.
Page 112
Zero-Touch Provisioning Deployment Scenarios Discovered units reload, and the former stack units join the Campus Fabric domain as PEs. Once the topology is active, verify the CSP operational state with the show spx csp all command, and confirm that the handshake is complete.
Page 113
Zero-Touch Provisioning Deployment Scenarios Once you confirm your selections, the new units reload and join the Campus Fabric domain as PE units. spx interactive-setup You can abort spx interactive-setup at any stage by <ctrl-c> 0: quit 1: change PE IDs 2: discover and convert new units (no startup-config flash) to PEs 3: discover and convert existing/new standalone units to PEs Please type your selection: 2...
Page 114
Zero-Touch Provisioning Deployment Scenarios Run the SPX interactive-setup utility on the CB to display the discovered topology. Once you confirm the topology with the selected PE IDs, the units reload and join the Campus Fabric domain. Select the topology to be included in the Campus Fabric domain and, if necessary, change the PE IDs assigned by default.
Page 115
Zero-Touch Provisioning Deployment Scenarios Once you confirm your selections, the new units reload and join the Campus Fabric domain as PE units. # spx interactive-setup You can abort spx interactive-setup at any stage by <ctrl-c> 0: quit 1: change PE IDs 2: discover and convert new units (no startup-config flash) to PEs 3: discover and convert existing/new standalone units to PEs Please type your selection: 2...
Zero-Touch Provisioning Deployment Scenarios In the following example, the stacking link 1/3/1 -2/3/1 has been removed. chain #0: Do you want to select this chain?(enter 'y' or 'n'): y New units form a ring. If you want to select all units, you must remove a link. Type the link to be removed: 0: no, 1: (#2 2/3 -- #1 2/2), 2: (#1 3/1 -- #2 3/1) (default: 0): 2 #1: icx7450-48f-sf-port-management-module CC4E.246C.F850 #2: icx7450-48f-sf-port-management-module CC4E.246D.10B8...
Page 117
Zero-Touch Provisioning Deployment Scenarios Unit Replacement Recommendations • Save the running configuration before replacing any unit. • Be sure that the replacement unit has the same model as the original and is connected the same way to the Campus Fabric domain. •...
Page 118
Zero-Touch Provisioning Deployment Scenarios Verify the topology using the show running-config command. SPX# show running-config Current configuration: ver 08.0.50b1T203 stack unit 1 module 1 icx7750-48-xgf-port-management-module module 2 icx7750-qsfp-6port-qsfp-240g-module priority 128 stack-port 1/2/1 1/2/4 stack unit 2 module 1 icx7750-48-xgf-port-management-module module 2 icx7750-qsfp-6port-qsfp-240g-module module 3 icx7750-6q-6port-qsfp-240g-module priority 128 stack-port 2/2/4...
Page 119
Zero-Touch Provisioning Deployment Scenarios Units 24 and 27 no longer appear in the topology. These are the units being replaced. SPX# show spx T=16m0.6: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address Pri State Comment S ICX7750-48XGF standby 609c.9f1f.9900 128 remote Ready S ICX7750-48XGF active cc4e.246d.9b00 128 local...
Page 120
Zero-Touch Provisioning Deployment Scenarios Run the SPX interactive-setup utility to discover the replacement PE units. Once the units are discovered, you are prompted for PE IDs. Retain the PE IDs. The topology is displayed again for confirmation. When you confirm the topology, the full topology, including the replacement PE units, is displayed.
Zero-Touch Provisioning Deployment Scenarios The replacement PEs reload and join the Campus Fabric domain. Use the show spx command to verify the operational state of the PEs, and then save the configuration. T=12m21.0: alone: standalone, D: dynamic cfg, S: static Type Role Mac Address...
Zero-Touch Provisioning Deployment Scenarios FIGURE 42 Using the spx zero-touch-deny Command • Configure spx zero-touch-deny in global configuration mode on a standalone unit. • To make units available for discovery again, enter the no zero-touch-deny command. The zero-touch enable or the spx pe-enable command also removes the spx zero-touch-deny setting.
Page 123
Zero-Touch Provisioning Deployment Scenarios FIGURE 43 Reference Topology: Changing PE IDs with SPX Interactive-Setup PE ID Change Considerations • The new PE ID can be assigned only if the ID is not taken and if no reserve configuration is associated with the ID. •...
Page 124
Zero-Touch Provisioning Deployment Scenarios In the following example, PE 17 becomes PE 24, and PE 18 becomes PE 23. Use the show running-config command to verify the domain and the PE ID configuration. ICX7750-48F Router(config)# show running-config Current configuration: ver 08.0.50b1T203 stack unit 1 module 1 icx7750-48-xgf-port-management-module module 2 icx7750-qsfp-6port-qsfp-240g-module...
Page 125
Zero-Touch Provisioning Deployment Scenarios Run the SPX interactive-setup utility, and choose option 1 to change PE IDs. When prompted, enter the desired PE IDs and confirm your changes. ICX7750-48F Router# spx interactive-setup You can abort spx interactive-setup at any stage by <ctrl-c> 0: quit 1: change PE IDs 2: discover and convert new units (no startup-config flash) to PEs...
Zero-Touch Provisioning Deployment Scenarios Use the show spx csp all or show debug all command to verify the CSP operational status and any other items as needed. If the topology includes a PE ring, use the show spx ring all command to verify that the ring is in the active state.
Page 127
Zero-Touch Provisioning Deployment Scenarios Identify the SPX port or SPX LAG on the CB where you want the PEs to be moved. In the example, PE 18 is moved to the new SPX port 2/1/2, and PE 19 is moved under PE 17 using existing SPX ports. Configure the required SPX port or SPX LAG on the CB where the PEs are intended to be moved.
Zero-Touch Provisioning Deployment Scenarios Deployment Scenario 8: Adding a New Link Between CB-PE or PE-PE to Change Existing Linear Toplogy to Ring Topology You can discover new physical links between a CB and an edge PE or between two edge PEs and form a PE ring from PE chains. This is applicable to FastIron 08.0.70 and later.
Page 129
Zero-Touch Provisioning Deployment Scenarios FIGURE 46 Adding New Links Between PE and PE SPX interactive-setup provides an option to discover new links and convert chains to a ring. In EXEC mode, enter the spx interactive-setup command, and select either option 2 or 3 when prompted. ICX7750-20Q Router# spx interactive-setup You can abort spx interactive-setup at any stage by <ctrl-c>...
Zero-Touch Provisioning Deployment Recommendations for ZTP After the process completes, enter the write memory command to save the newly generated SPX port and SPX LAG configuration. Probing topology to find new units... Detect 2 new links (classified to 1 logical link) between existing units. New logical link 0: ======================================================= +----+ +----+ +----+ +----+ 2/1/9--4/1| 23 |2/1==2/1| 24 |2/5==2/1| 25 |2/5==2/5| 26 |...
Zero-Touch Provisioning Best Practices • Multiple links between any two units cause loops until they are converted to an SPX LAG. Ruckus recommends using spx-port or spx-lag to avoid loops. • Loops cause high traffic in the looped ports and may cause high CPU if the looped packets are trapped to the CPU. •...
Zero-Touch Provisioning Debugging Notes • Do not attempt to change PE IDs when spx allow-pe-movement is configured. Ruckus recommends changing PE IDs, if necessary, during the initial bring-up of Campus Fabric, before connecting hosts or configuring any protocols. • If ports are planned for dedicated use in a Campus Fabric domain, configure the ports as SPX LAGs or SPX ports rather than as zero-touch ports.