Part No. 031496-00, Rev. C September 2005 OmniSwitch 6624/6648/ 6800/7700/7800/8800 Troubleshooting Guide www.alcatel.com OmniSwitch Troubleshooting Guide September 2005...
Before Calling Alcatel’s Technical Assistance Center ... xx Chapter 1 Troubleshooting the Switch System In This Chapter ...1-1 Introduction ...1-2 Troubleshooting System for OS-6624/6648 and OS-7/8XXX ...1-3 Advanced Troubleshooting ...1-9 Dshell Troubleshooting ...1-11 Troubleshooting NIs on OmniSwitch 7700/7800/8800 ...1-21 OmniSwitch 6624/6648 Dshell Troubleshooting ...1-23 Accessing Dshell on Idle Switches ...1-25...
Page 4
Verify Source Learning ...2-6 Verify Switch Health ...2-7 Verify ARP ...2-7 Using the Log File ...2-8 Checking the 7700/7800 Nantucket Fabric ...2-8 Checking the 7700/7800 Nantucket Fabric for Interrupts, Data Counts and Error Counts ...2-9 Checking the Traffic Queue on the NI ...2-9 Check for Catalina (MAC) or Port Lockup ...2-10 Chapter 3 Troubleshooting Source Learning...
Page 6
DNS Configuration Considerations ...6-3 Chapter 7 Troubleshooting Link Aggregation In This Chapter ...7-1 Link Aggregation Limits and Guidelines ...7-2 OmniSwitch 6624/6648 Restrictions ...7-2 Troubleshooting a Link Aggregation Failure ...7-3 Verify the Configuration ...7-3 Source Learning ...7-5 Link Aggregation Affecting Other Traffic ...7-5 Problems Creating a Group ...7-5...
Page 8
Dshell Troubleshooting ...11-8 Viewing the ARP Table on OmniSwitch 6624/6648 Switches ...11-10 Chapter 12 Troubleshooting IP Routing In This Chapter ...12-2 Introduction ...12-3 IP Routing Protocol Failure ...12-3 Troubleshooting via the CLI ...12-3 Troubleshooting with Debug CLI ...12-11 RIP Troubleshooting ...12-13 OSPF Troubleshooting ...12-19...
Page 9
Dshell Troubleshooting ...14-9 Chapter 15 Troubleshooting DVMRP In This Chapter ...15-1 Introduction ...15-2 DVMRP Troubleshooting ...15-2 DVMRP Global and Interface Commands ...15-2 DVMRP Debug Commands ...15-4 Chapter 16 Troubleshooting PIM-SM In This Chapter ...16-1 Introduction ...16-2 Definition of Terms ...16-2 Protocol Overview ...16-3 DR Election ...16-3 Simplified Hello Message Format ...16-3...
Page 10
Multicast Group Status is Shown as Disabled ...16-27 PIM-SM Limitations ...16-28 Upstream Neighbor/Next Hop Debug Commands ...16-28 Chapter 17 Troubleshooting Server Load Balancing In This Chapter ...17-1 Introduction ...17-2 Server Load Balance Failure ...17-2 What is an SLB Failure? ...17-2 Description of a Complete Failure of Service ...17-2 Description of a Partial Failure of Service ...17-2 Troubleshooting Commands ...17-3...
Page 11
Chapter 19 Troubleshooting 802.1X In This Chapter ...19-1 Troubleshooting with the CLI ...19-2 Troubleshooting Using Debug CLI ...19-4 Dshell Troubleshooting ...19-7 Appendix A OS6600/OS7700/OS8800 Architecture Overview In This Chapter ... A-1 The MAC ASIC ... A-2 Catalina ... A-2 Firenze ... A-4 The Coronado ASIC ...
Page 12
Calendar Manager Module ... A-19 Data Port Output Module ... A-19 Nantucket Redundancy ... A-19 Roma ... A-22 Functional Description ... A-23 Initialization ... A-24 NI Slot Insertion ... A-25 Setup Calendars and Flow Control for New NI ... A-25 NI Slot Extraction ...
• OmniSwitch 8800 • Note. All references to OmniSwitch 6624 and 6648 switches also apply to the OmniSwitch 6600-P24, OmniSwitch 6600-U24, OmniSwitch 6602-24, and OmniSwitch 6602-48 unless specified otherwise. Unsupported Platforms The information in this guide does not apply to the following products: OmniSwitch (original version with no numeric model name) •...
Note. Troubleshooting documentation for legacy products (e.g., Omni Switch/Router) can be downloaded at http://support.ind.alcatel.com/releasefiles/indexpage.cfm. Who Should Read this Manual? The principal audience for this user guide is Service and Support personnel who need to troubleshoot switch problems in a live network. In addition, network administrators and IT support personnel who need to configure and maintain switches and routers can use this guide to troubleshoot a problem upon advice from Alcatel Service and Support personnel..
The following are the titles and descriptions of all the Release 5.1 and later OmniSwitch user guides: OmniSwitch 6600 Family Getting Started Guide • Describes the hardware and software procedures for getting an OmniSwitch 6624 or 6648 up and running. Also provides information on fundamental aspects of OmniSwitch software and stacking architecture.
Page 18
Network Interface (NI) modules. OmniSwitch CLI Reference Guide • Complete reference to all CLI commands supported on the OmniSwitch 6624/6648, 7700/7800, and 8800. Includes syntax definitions, default values, examples, usage guidelines and CLI-to-MIB variable mappings. OmniSwitch 6600 Family Switch Management Guide •...
Page 19
OmniSwitch 7700/7800/8800 Network Configuration Guide • Includes network configuration procedures and descriptive information on all the major software features and protocols included in the base software package. Chapters cover Layer 2 information (Ethernet and VLAN configuration), Layer 3 information (routing protocols, such as RIP and IPX), security options (authenticated VLANs), Quality of Service (QoS), link aggregation, and server load balancing.
Before Calling Alcatel’s Technical Assistance Center Before calling Alcatel’s Technical Assistance Center (TAC), make sure that you have read through the appropriate section (or sections) and have completed the actions suggested for your system’s problem. Additionally, do the following and document the results so that the Alcatel TAC can better assist you: Have a network diagram ready.
• tion Guide is highly recommended. In This Chapter “Introduction” on page 1-2 “Troubleshooting System for OS-6624/6648 and OS-7/8XXX” on page 1-3 “Advanced Troubleshooting” on page 1-9 “Dshell Troubleshooting” on page 1-11 “Using AlcatelDebug.cfg” on page 1-26 “Troubleshooting IPC on OS-6/7/8XXX Series of Switches” on page 1-27 “Port Numbering Conversion Overview”...
Introduction Introduction The CMM is the Management Module of the switch. All of the critical operations of the switch including the monitoring is the responsibility of the CMM. CMM not only provides monitoring but also synchro- nizes all of the NI for different operations. The operation of the CMM is the same in OS-6/7/8XXX switches.
Verify that all of the modules in the chassis are up and operational, using the command: -> show module status Operational Slot Status ------+-------------+------------+---------+----------------- CMM-1 CMM-2 SECONDARY NI-1 OmniSwitch Troubleshooting Guide Troubleshooting System for OS-6624/6648 and OS-7/8XXX Firmware Admin-Status POWER ON 00:d0:95:6b:09:40 POWER ON 00:d0:95:6b:22:5c POWER ON 00:d0:95:6b:23:2e POWER ON 00:d0:95:6b:3a:20...
Page 24
Troubleshooting System for OS-6624/6648 and OS-7/8XXX NI-2 NI-3 NI-4 NI-5 NI-6 NI-7 NI-8 OmniSwitch 6600 with 8 stackable switches show up. Notice that the switch with ID 1 is the primary CMM and the switch with ID of 2 is the secondary. All the switch also show up as NI because each switch has a CPU and is also a NI.
Page 25
Power Supplies in chassis 6 Operational Status -----+------------------- PS-1 PS-2 NOT PRESENT Power Supplies in chassis 7 Operational Status -----+------------------- PS-1 PS-2 NOT PRESENT OmniSwitch Troubleshooting Guide Troubleshooting System for OS-6624/6648 and OS-7/8XXX B42N101P2, OCT 18 2001, POWER ON, September 2005 page 1-5...
Page 26
If none of the above are above the threshold then the next step is to try to isolate the problem to a particu- lar NI (or a switch within an OmniSwitch 6624/6648 stack) with the show health slot_number CLI command: ->...
Page 27
NI slot 1, GBIC port 2 MON AUG 21 23:28:33 2023 d on NI slot 1, GBIC port 1 MON AUG 21 23:28:33 2023 OmniSwitch Troubleshooting Guide Troubleshooting System for OS-6624/6648 and OS-7/8XXX Application Level Log Message HSM-CHASSIS...
Page 28
Troubleshooting System for OS-6624/6648 and OS-7/8XXX d on NI slot 1, GBIC port 2 MON AUG 21 23:28:39 2023 ed on NI slot 5, GBIC port 2 MON AUG 21 23:30:39 2023 d on NI slot 5, GBIC port 2...
Troubleshooting the Switch System Advanced Troubleshooting One level of switch logging is stored in the two log files located in the /flash directory. There is another low level debug that can be enabled and used for diagnosing the problems. This debug is known as “systrace”, meaning system trace.
Page 30
Advanced Troubleshooting _SM_IPCUP_TIMEOUT 3349118948 CSM-CH info 3345200526 CSM-CH info 3342928783 CSM-CH info 3342928661 CSM-CH info _SM_IPCUP_TIMEOUT 3342928628 CSM-CH info 3336738410 CSM-CH info 3336738287 CSM-CH info _SM_IPCUP_TIMEOUT 3336738256 CSM-CH info 3334849145 CSM-CH info 3330548020 CSM-CH info 3330547902 CSM-CH info _SM_IPCUP_TIMEOUT 3330547869 CSM-CH info 3324495309 CSM-CH info 3324357940 CSM-CH info 3324357816 CSM-CH info...
Troubleshooting the Switch System Dshell Troubleshooting To further diagnose the task consuming the CPU on the CMM one needs to use the following Dshell commands: Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
Page 33
Troubleshooting the Switch System 2 tasks were created. 2 tasks were deleted. spyStop value = 0 = 0x0 It seems that the CPU task is high because of tNetTask, Ipedr, and tOSPF. Check to see if any of the task is suspended on the CMM. Working: [Kernel]->i NAME ENTRY...
Page 34
Dshell Troubleshooting 490f9e0 490f920 value = 76612560 = 0x49103d0 Certified: [Kernel]-> To troubleshoot a CPU or memory spike with 5.1.5.X, you can start a software routine in dshell and it will log the task name to the swlog whenever there is a spike in CPU or memory usage. Switch/>...
Troubleshooting the Switch System tCS_CVM SesMon_12 tTelnetOut0e4208c tTelnetIn0 CliShell0 tPolMonSvr tDcacheUpd To further qualify the source of the problem we need to look at each and every NI. Troubleshooting NIs on OmniSwitch 7700/7800/8800 Looking at the health statistics of each NI would give an idea about which one is causing the problem. Following CLI command can be used to diagnose: Show health <slot number>...
Troubleshooting the Switch System OmniSwitch 6624/6648 Dshell Troubleshooting One of the important things in OS-6600 is to confirm the stack topology. This can be confirmed using the command: Working: [Kernel]->smctx **************************************************** local_slot=1 * base_mac= 00:d0:95:84:4b:d2 * local_mac=0000 1111 1111 * TYPE_48...
Page 44
Dshell Troubleshooting output definitions (continued) Local Mac Role In_loop Neighbor1 Topology Role Topology Outport Topology base Mac Lookup Gport Lport Status To view the stack topology in detail, use the following command: Working: [Kernel]->stack_topo local_slot=1 role=PRIMARY P=1 S=2 (elements=0x300ff nb=8 loop=1 sup=2 type=2) 7 elements seen by link1 (gport=0x1a lport=0x1a status=1) slot=8 originate_port=26 role=IDLE slot=7 originate_port=27 role=IDLE...
Using AlcatelDebug.cfg In stacking environment, only the primary and secondary switches have console enabled whereas console is disabled for the idle switches. To enable the Dshell access to the idle switches use the following command on primary stack: Nisup_control_WW_on slot You must execute this command on each idle switch in the stack.
Troubleshooting the Switch System Troubleshooting IPC on OS-6/7/8XXX Series of Switches IPC (Inter Process Communication) is should by the system to communicate between different software modules. This communication can be between different processes in the same software module or between two entirely separate modules.
Page 48
Troubleshooting IPC on OS-6/7/8XXX Series of Switches LocalPool: Full size is 64, remaining: 64 In socket queues: 0 Not queued: 0: In DMA queues: 0 Each type of pool has the following listed in the command output: Maximum size of buffers available •...
Page 49
Troubleshooting the Switch System LocalPool: Full size is 64, remaining: 64 In socket queues: 0 Not queued: 0: DMA queues: 0 In the above two outputs it seems that the control pool is stuck and the socket queues are incrementing. In order to find out which task is using these queues we need to look at the socket information.
Page 50
Troubleshooting IPC on OS-6/7/8XXX Series of Switches value = 10 = 0xa Working: [Kernel]-> The above command displays a lot of information but we are interested in the most repeating socket ID. In the above example it is 0x8. To look into what does this socket means the following command can be used in Dshell: Working: [Kernel]->ipc_socket_info 0x8 ipc_socket_info...
Page 51
Troubleshooting the Switch System Now doing a task trace on this task can be helpful to see if the task is moving: Working: [Kernel]->tt 0x67f3c10 108e9c vxTaskEntry 00, 6a69800) 66b69b4 Letext 6ff56f8 zcSelect 158b4c semTake ke, 264c00) value = 0 = 0x0 Working: [Kernel]->...
Page 52
Troubleshooting IPC on OS-6/7/8XXX Series of Switches LocalPool: Full size is 0, remaining: 1024 In socket queues: 0 Not queued: 0: In DMA queues: 0 value = 6 = 0x6 The above display of the command does not show the Full size of any of the pools. This indicates that CMM is unable to view the IPC pools of the NI.
Page 53
Troubleshooting the Switch System 631908 next = 0x17c8c44, pFreeQ = 0x2fc7a8, data_offset = 0, free_list_num = 6 Socket ID = 0x19, dest slot = 2, remote addr = 0x3030002, ipc status = S Task ID = 0x17fd170, PayLoad Len= 128, 632108 next = 0x17caab0, pFreeQ = 0x2fc7a8, data_offset = 0, free_list_num = 6 Socket ID = 0x19, dest slot = 2, remote addr = 0x3030002, ipc status = S...
OmniSwitch 6624/6648 Example Follow the steps below for an example of displaying IPC pool data on an OmniSwitch 6624/6648;r Check the In socket queues and Not queued fields for all the pools and identify the pool that has the highest value with the ipc_pools command as shown below: Working: [Kernel]->ipc_pools...
Page 55
Troubleshooting the Switch System Socket ID = 0x100, dest slot = 5, remote addr = 0x5400042, ipc status = S Task ID = 0x7571700, PayLoad Len= 812, next = 0x739b810, pFreeQ = 0x74fb4e0, data_offset = 0, free_list_num = 6 Socket ID = 0x100, dest slot = 65, remote addr = 0x8440041, ipc status = S Task ID = 0x7571700, PayLoad Len= 812, next = 0x7396da4, pFreeQ = 0x74fb4e0, data_offset = 0, free_list_num = 6 Socket ID = 0x2, dest slot = 65, remote addr = 0x11b0001, ipc status = G...
Port Numbering Conversion Overview Port Numbering Conversion Overview The sections below document how to convert port number parameters. Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
OmniSwitch 6624/6648 (Hawk) Example Find all valid lports values with the dmpValidPorts command from Dshell on each element (i.e., each slot in a stack). Afterwards, you should do a dump for each slot. Use the dmpValidPorts command as shown below: Certified: [Kernel]->dmpValidPorts...
Page 58
Port Numbering Conversion Overview Troubleshooting the Switch System page 1-38 OmniSwitch Troubleshooting Guide September 2005...
2 Troubleshooting Switched Ethernet Connectivity This chapter assumes that it has been verified that the connectivity problem is across Ethernet media and the connection between the non-communicating devices is switched/bridged not routed (i.e., Devices are in the same IP Subnet). For configuration assistance in designing and configuring switched Ethernet connectivity, please refer to the “Configuring Ethernet Ports”...
Overview of Troubleshooting Approach Overview of Troubleshooting Approach Verify physical layer connectivity. • Verify current running configuration is accurate. • Verify source learning. • Investigate any error conditions. • Verify health of NIs involved. • Verify health of CMM. • Client A IP = 192.168.10.2 page 2-2...
Troubleshooting Switched Ethernet Connectivity Verify Physical Layer Connectivity Verify that there is valid link light along the entire data path between the devices that can not switch to each other. Make sure to include all interswitch links. Verify LED’s on all involved CMMs and NIs are Solid OK1, Blinking OK2.
Page 62
Verify Physical Layer Connectivity -> show ni 5 Module in slot 5 Model Name: Description: Part Number: Hardware Revision: Serial Number: Manufacture Date: Firmware Version: Admin Status: Operational Status: Power Consumption: Power Control Checksum: MAC Address: ASIC - Physical: -> show cmm Module in slot CMM-A-1 Model Name: Description:...
Troubleshooting Switched Ethernet Connectivity Verify Current Running Configuration If the physical layer looks OK, then verify the configuration. Use the show configuration snapshot all to display the current running configuration. Use this command to verify the ports that are involved are in the correct VLAN.
Verify Source Learning To further verify the ports are in the correct VLAN and that they are in spanning tree forwarding instead of blocking use the show vlan port command. Also note that the port type must match what it is connecting to.
Troubleshooting Switched Ethernet Connectivity Verify Switch Health If source learning appears to be not working correctly, verify the health of the switch with the show health, and show health slot commands. Be sure to run the latter command on all necessary NIs. Any variables that have reached or exceeded their limit value could cause forwarding problems on the switch.
Using the Log File Using the Log File If none of the above suggest a reason as to why Ethernet switching is not properly working, look into the log file and see if there are any messages that may suggest why switching is not working properly. Use the show log swlog command to view the system log file.
Troubleshooting Switched Ethernet Connectivity Checking the 7700/7800 Nantucket Fabric for Interrupts, Data Counts and Error Counts Working: [Kernel]->nanListB02 HB Out of Sync Interrupts: 0 Error Count Exceeded Interrupts: 0 Framing Error Interrupts: 0 Parity Error Interrupts: 0 B02 Data Port 0 Frame Count = 690dbd37 B02 Data Port 1 Frame Count = 0 B02 Data Port 2 Frame Count = 542e70d9 B02 Data Port 3 Frame Count = 0...
3 Troubleshooting Source In order to troubleshoot Source Learning problems, a basic understanding of the process is required. A review of the “Managing Source Learning” chapter from the appropriate OmniSwitch Network Configu- ration Guide is required. The following RFC and IEEE standards are supported: RFCs supported IEEE Standards supported In This Chapter...
Introduction Introduction VLAN 114 Port: 8/23 IP: 10.40.114.50 MAC:00-C0-4f-12-F7-1B When a packet first arrives on NI source learning examines the packet and tries to classify the packet to join its correct VLAN. If a port is statically defined in a VLAN, the MAC address is classified in the default VLAN.
Troubleshooting Source Learning Troubleshooting a Source Learning Problem In order to troubleshoot a source learning problem the first step is to verify that the physical link is up and the port has correctly auto-negotiated with the end-station. The next thing is to verify that the port is a member of the right VLAN, if a port is statically configured for a VLAN, or the Group Mobility policies are correctly defined.
Page 72
Troubleshooting a Source Learning Problem In order to narrow down to a specific NI the following command can be used (any valid slot number can be specified): -> show mac-address-table slot 8 Legend: Mac Address: * = address not valid Vlan Mac Address ------+-------------------+--------------+-----------+------------+-----------...
Troubleshooting Source Learning Once, the MAC addresses are learned on the ports then the devices should be able to communicate depending on the upper layers. Variations of MAC-related commands can be viewed in the “Managing Source Learning” chapter from the appropriate OmniSwitch Network Configuration Guide. Advanced Troubleshooting The advanced troubleshooting for Source learning related problems is to look whether the traffic is coming in from a port and the NI is not learning the MAC, if not prevented by using any other rules.
Page 74
Advanced Troubleshooting format text format all output screen output switchlog board cmm board ni board all board none If the problems are associated with the source learning on a specific NI then the limitations of the Number of MAC addresses learned should also be considered. Current limitations are: Number of learned MAC addresses per network interface (NI) module...
Troubleshooting Source Learning Dshell Troubleshooting The OmniSwitch 6/7/8XXX has a distributed architecture. Source Learning is specific to a NI. Each NI has a layer 2 pseudo-cam which is which can hold 64K entries. 32K entries are reserved for L2 Source Addresses which are local to that NI in L2SA table and the rest of 32K entries are reserved for L2 Destina- tion Addresses which can be from local or remote NI in L2DA table.
Page 76
Dshell Troubleshooting 000001f3 Total L2 SA entry amount = 1 Both of the MAC addresses are learned in the correct VLANs on the right NI. Now, if device A is trying to communicate to device B then the next thing to look for is the destination MAC address table.
Dshell Troubleshooting OS-6600 To look at the forwarding database on OS-6600 in Dshell use the slcDumpSlotSlice command., which displays which slot/slice is considered to be up and operational by the source learning software: Certified: [Kernel]->slcDumpSlotSlice Source Learning Slice Up List: slot/slice 2/0, type = 838930434, firstgport = 64, lastgport = 123 value = 68 = 0x44 = ’D’...
Page 79
Troubleshooting Source Learning L2 Physical Pool Stats: DstSwp Tables NetID Tables Protocol Tables ASIC Rsrc Wraps value = 294 = 0x126 Output of many fields are described below: output definitions Addr Addr AVID DstSwp Tables NetID Tables To see Source learning in action, set the debug level higher (levels are 1-6): SlnDebugLevel=1 The following is a sample output: Certified: [Kernel]->SlnDebugLevel=1...
4 Troubleshooting Spanning In order to troubleshoot spanning tree related problems an understanding of the protocol and its features are needed. The OmniSwitch supports two Spanning Tree Algorithms; 802.1D (standard) and 802.1w (rapid reconfiguration). In addition, the Omniswitch supports two Spanning Tree operating modes: flat (single STP instance per switch) and 1x1 (single STP instance per VLAN).
Troubleshooting Spanning Tree Troubleshooting Spanning Tree A failure of the Spanning Tree Protocol (STP) will usually cause either a bridge loop on the LAN or constant reconvergence of STP. This in turn can cause several resultant problems. If there is a bridge loop on the LAN, there can appear to be a broadcast storm since broadcast packets •...
Page 83
Troubleshooting Spanning Tree Use the show spantree ports command to determine if the port is in forwarding or blocking and are in the correct VLAN. Remember that in any LAN with physical redundancy there must be at least one port in blocking status.
Page 84
Troubleshooting Spanning Tree Since STP is run in a distributed fashion it is important to verify that each NI that is involved is not having a resource problem. Use the show health command to verify the resources available on an NI. ->...
Troubleshooting Spanning Tree Dshell As mentioned previously, it is important to verify the health of the NI as well as the CMM. Please refer to Chapter 1, “Troubleshooting the Switch System,” Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
Page 86
Dshell Working: [Kernel]->esmDumpCoronado 1,0,0x6608000+140*4,32 6608230 : 6608250 : 6608270 : 6608290 : value = 1 = 0x1 Working: [Kernel]->esmDumpCoronado 1,0,0x6608000+511*4,32 66087fc : 660881c : 660883c : 660885c : value = 1 = 0x1 The above commands that the spanning tree vector is set for Gigabit port 1/1 for VLANs 1, 140, 141, 150, and 511.
Page 87
Troubleshooting Spanning Tree 660885c : value = 1 = 0x1 Binary: 0000 0000 0011 For VLAN 1 the bits set are 203 which are equivalent to binary 0000 0000 0011. Bits 1 and 2 are set indi- cating that ports 1 and 2 have the spanning tree vector set for VLAN 1. The next register value is for VLAN 2, hex value is 8000000.
Page 88
Dshell 1:0 Message age of received BPDU 1:0 PIM port c state 5 1:0 recordProposed operPointToPointMAC=1 1:0 PIM port c state 7 1:0 PIM port c state 4 1:0 port 12 is forward (5) 1:0 tick (tack) time is now 701603 1:0 RSTBPDU transmitted on port 33 Root bridge ID = 3200d0 95820514 Path to Root cost = 3...
Page 89
Troubleshooting Spanning Tree Designated bridge ID = 800000d0 957962aa Designated portId = 29697 Bridge portId = 29697 Message age Proposing 1:0 tick (tack) time is now 701647 1:0 tick (tack) time is now 701648 1:0 RSTBPDU transmitted on port 33 Root bridge ID = 3200d0 95820514 Path to Root cost = 3 Designated bridge ID = 800000d0 957962aa...
Generic Troubleshooting in Dshell Generic Troubleshooting in Dshell Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
Generic Troubleshooting in Dshell MODVLADM This event is received is sent by the Spanning Tree CMM to the NIs when the administrative state of a VLAN is changed (event generated by the VLAN Manager to the Spanning Tree CMM). The parameters are: First parameter: The VLAN identifier.
Troubleshooting Spanning Tree MDEFVLAN This event is received is sent by the Spanning Tree CMM to the NI when the default VLAN of a fixed or q-tagged port is change (this also applies to logical port). This event is generated on the CMM by VLAN Manager application.
Generic Troubleshooting in Dshell PORTJOIN This event is sent by Link Aggregation NI when a physical port is joining an aggregator; It could be either a static aggregator (OmniChannel) or a dynamic aggregator (802.3ad). This message is generated after the first port has joined the aggregator (see The parameters are: First parameter: The aggregator identifier (logical port ID value between 0 and 31).
Troubleshooting Spanning Tree PTSTPMOD The is event is generated by the Spanning Tree CMM when the Spanning Tree configuration parameter of a port is changed by the operator. The parameters are: First parameter: The spanning identifier (i.e., VLAN identifier). • Second parameter: The global port identifier.
Generic Troubleshooting in Dshell The Port VLAN blocking message sent to the Source Learning NI has the following structure: uint16 VlanId, uint32 PortVector This event has the following values for the message ID: appID: APPID_SPANNING_TREE. • subMsgNum: STP_PortVlanBlocking. • These event fields are defined below: VlanId: A value 1 to 4095 identifies a VLAN (0 means that the message is applied to ports defined by •...
Troubleshooting Spanning Tree GM2FIXED This message is sent by Group Mobility NI in response to a BPDU on mobile port message sent by the Spanning Tree. By sending this message group mobility tells to Spanning Tree that the mobile port must be reverted to the fixed state.
Generic Troubleshooting in Dshell TOPOCHGT This event notifies a change of Spanning Tree topology. The format of the message is: uint16 VlanId, uint16 aging_timer This event has the following values for the message ID: appID: APPID_SPANNING_TREE • subMsgNum: STP_TopologyChange • These event fields are defined below: VlanId: A value of 1 to 4095 identifies a VLAN and 0 means that the message is applied to all the •...
Troubleshooting Spanning Tree Physical and Logical Port Dumps Logical Ports (stpni_debugLport) Here follows the display of the Logical port seen by the Spanning Tree. Each line corresponds to the local port identifier index. Certified: [Kernel]->stpni_debugLport Logical Ports array: sta field: - 0x80 ->...
Page 100
Generic Troubleshooting in Dshell The fields displayed by the stpni_debugLport command are described below: output definitions dGid Portid NTag Vector Prim Duplex Physical Port (stpni_debugPport) Here is the display of the Physical Port seen by the Spanning Tree NI: Certified: [Kernel]->stpni_debugPport Physical Ports array: sta field: - 0x80 ->...
Generic Troubleshooting in Dshell output definitions (continued) Portid NTag Vector lpid Prim Duplex Physical and Logical Port Trace Display (stpni_debugport) This is a combination of the Logical and Physical port display. See page 4-19 “Physical Port (stpni_debugPport)” on page 4-20 Socket Handler Traces These traces include Global data, warning, and event traces.
Troubleshooting Spanning Tree sockGlobal_maxmsgevt: This counter is the maximum number of message (counter) processed • consecutively on the Message Event Manager channel (inter-NI STP channel). sockGlobal_looptick: This flag is set to 1 indicates that we loop until the maximum number of •...
Generic Troubleshooting in Dshell Inter-NI Trace (stpNISock_intraceprint) This trace records all the inter-NI STP events received by the Socket Handler and has the following format: An ASCII pattern reflecting the event. • Up to 4 parameters (a -1 (or 0xffffffff) indicates that the parameter is not significant). •...
Troubleshooting Spanning Tree CMM Spanning Tree Traces Trace Menu The stpCMMSock_help Dshell command displays the Spanning Tree Manager menu as shown below. -> dshell Certified: [Kernel]->stpCMMSock_help Slot : 65 Slice : 0 stpCMMSock_globals stpCMMSock_warningprint stpCMMSock_totraceprint stpCMMSock_traceprint stpCMMSock_ttimetraceprint : treatment time trace stpCMMSock_boardupprint stpCMMSock_printon stpCMMSock_printoff...
Writing a PR for Spanning Tree Writing a PR for Spanning Tree The following subsections describe some guidelines to follow when writing a PR that addresses Spanning Tree. Please note that the following subsections use Dshell commands, not CLI commands. Note.
Troubleshooting Spanning Tree Spanning Tree Unchanged When Port State Has Changed If the show spanning tree CLI command still displays the same information while a port state has changed then the problem could be due to a broken communication path between the CMM and NI. In this case do the following (for both the CMM and NI): Time-out trace of the socket handler (stpNISock_totraceprint or stpCMMSock_totraceprint).
Page 108
Writing a PR for Spanning Tree Troubleshooting Spanning Tree page 4-28 OmniSwitch Troubleshooting Guide September 2005...
5 Troubleshooting BOOTP/ In order to troubleshoot a BOOTP/DHCP and UDP Relay, a basic understanding of the protocol is required. Some basic concepts are covered below. The OmniSwitch supports UDP Relay. Reading the “DHCP Relay” chapter from the appropriate OmniSwitch Network Configuration Guide is also highly recommended.
Starting the Troubleshooting Procedure Use a Network Diagram It is extremely important to know where the server is in relation to the client, which switch both the client and the server is directly connected to and their port numbers. A network diagram presents this kind of information, for example, in an easily understood matter.
Troubleshooting BOOTP/DHCP/UDP Relay Forward Delay Timer Forward delay is the amount of time in seconds UDP relay will wait before forwarding a request to a DHCP server, or the same DHCP server. (If only one is configured on the switch.) This allows the DHCP server who initially got the DHCP request packet from the client to respond before the request is forwarded to another DHCP server.
Starting the Troubleshooting Procedure UDP Relay and Group Mobility If UDP Relay is being used with a Mobile DHCP Rule, determine if the end station is simply getting the wrong IP address scope. If this is the case, verify group mobility, as the source VLAN of the UDP request could be wrong when it reaches the UDP Relay function.
Troubleshooting BOOTP/DHCP/UDP Relay Advanced Troubleshooting for UDP Relay To monitor the UDP traffic, the debug ip packet protocol udp CLI command can be used. Note. See the “IP Commands” chapter in the OmniSwitch CLI Reference Guide for more information. The output below shows the entire conversation of a DHCP client with MAC address 000039:73130 in VLAN 20 to a DHCP server in VLAN 10 with a IP address of 10.10.10.58.
Dshell Dshell To send the UDP Relay debug to the console, follow the following commands: Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
Page 115
Troubleshooting BOOTP/DHCP/UDP Relay In the example above, the BOOTP request was dropped because the number of seconds elapsed since the start of the process was not incremented (secs=0), and the forward delay was set to 3 seconds. We got in a tweaked zero IP address packet on BSD socket shown below: Recvd on bsd socket pkt from 0.0.0.0, rtr-port addr=20.20.20.200, 0x336860360 Received request packet for the bootp service on bsd socket hops=0x0...
6 Troubleshooting DNS In order to troubleshoot a DNS problem, a basic understanding of the protocol/feature is required. Some basic concepts are covered below. Reading the “Enabling the DNS Resolver” section in the “Logging Into the Switch” chapter in the appropriate OmniSwitch Switch Management Guide is highly recommended. In This Chapter “Introduction”...
Troubleshooting a DNS Failure Troubleshooting a DNS Failure Starting the Troubleshooting Procedure If you try to use DNS resolution and it does not resolve, or connect from the switch with error such as “unknown host” take the following steps. Verify IP connectivity from the switch in question to the DNS server by pinging the server (destination) in question from the switch (source) by its IP address.
Troubleshooting DNS If you find conflicting information, then see which portion has the wrong information and focus on that • layer again (layer 2, 3, or 7). The response packet should contain the following fields: DLC with the source Mac address of the DNS •...
Page 120
DNS Configuration Considerations Troubleshooting DNS page 6-4 OmniSwitch Troubleshooting Guide September 2005...
7 Troubleshooting Link In order to troubleshoot a Link Aggregation issue a basic understanding of the protocol is required. Read- ing the “Configuring Static Link Aggregation” and “Configuring Dynamic Link Aggregation” chapters in the appropriate OmniSwitch Network Configuration Guide is also highly recommended. The OmniSwitch supports two Link Aggregation Algorithms: Two methods exist for configuring Link Aggregation: Static Link Aggregation Groups—Also referred to as OmniChannel used for Aggregation of Multi-...
In addition, ports must be configured sequentially and the first port configured must begin with port number 1, 9, 17, or 25 on an OmniSwitch 6624 or 1, 9, 17, 25, 33, 41, 49, or 51 on an OmniSwitch 6648.
Troubleshooting Link Aggregation Troubleshooting a Link Aggregation Failure Switch A OmniSwitch 7800 The figure above has the following setup: Switch A and Switch B connected back to back and Link Aggregation configured. • Port 7/1 of Switch A is connected to port 7/1 of Switch B. •...
Page 124
Troubleshooting a Link Aggregation Failure The show linkagg port CLI command followed by the slot and port number will display the port and link state as well as if it is the primary port. (See the samples below.) If the operational or administrative state is down and the port is primary, this indicates a software problem.
There are a maximum of 32 aggregate groups allowed on an OmniSwitch 7700/7800/8800switch, 30 • on a stack of OmniSwitch 6624/6648 switches, and 8 on a single OmniSwitch 6624/6648 switch. The number goes down on OmniSwitch 7700/7800/8800 switches depending on the size of the aggregate, see below.
Troubleshooting a Link Aggregation Failure LACP 802.3AD Most of the steps followed previously in this chapter apply to troubleshooting LACP. To verify the config- uration use the show linkagg [agg. Number]. Again, verify the aggregate is enabled and up. The most important aspect in troubleshooting LACP is to verify the transmission of valid LACPDUs. For this you can go to the Dshell section as well as use a sniffer tool.
Troubleshooting Link Aggregation Advanced Link Aggregation Troubleshooting To verify that the link aggregate software recognizes all the available slices, perform the following steps: Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
Page 129
Troubleshooting Link Aggregation agg_multicast_frm_tx_ok agg_broadcast_frm_tx_ok agg_unicast_frm_tx_ok agg_frm_discard_tx agg_frm_with_tx_errors value = 40000003 = 0x2625a03 Since LACP is run on the NI it is important to verify the NI has the proper information. Many of the same commands run above are available in the NI debugger. (See the table below.) The syntax and output are the same.
6800 Link Aggregation Debug Functions 6800 Link Aggregation Debug Functions The following functions are available for Link Aggregation debugging on the NI. A summary is shown below. Display Aggregates : la_ni_agg_prt Display Ports : la_ni_port_prt Display Ports Up : la_ni_port_up_prt Display LACP stats : la_ni_lacp_port_stats_prt...
6800 Link Aggregation Debug Functions la_ni_trace_freeze Certified: [Kernel]->la_ni_trace_freeze value = 244315184 = 0xe8ff430 la_ni_trace_unfreeze Certified: [Kernel]->la_ni_trace_unfreeze #_LA_SRV_TRACE_NI_LINKAGG_TRAC_UNFROZEN value = 40 = 0x28 = '(' la_ni_kite_help The la_ni_kite_help function may be accessed from the D-shell and will display the following output: Working: [Kernel]->la_ni_kite_help la_ni_kite_get_bcm_trunk_info(unit,agg_id) value = 83 = 0x53 = 'S'...
8 Troubleshooting 802.1Q In order to troubleshoot an 802.1Q problem on a port, a basic understanding of the networking OSI model is required to assist one with the troubleshooting steps to resolve a particular network problem. Alcatel’s OmniSwitch supports 802.1Q specifications as defined by RFC 2674/IEEE 802.1Q/D11. Reading the “Configuring 802.1Q”...
Troubleshooting 802.1Q Troubleshooting 802.1Q When troubleshooting an 802.1Q problem, it is important to not only investigate the 802.1Q feature and configuration, but also the basic Ethernet connectivity between the problematic switches. Please refer to Chapter 2, “Troubleshooting Switched Ethernet Connectivity,” Switch A OmniSwitch 7800 If there is no traffic passing at all across an 802.1Q link, verify basic Ethernet connectivity.
Troubleshooting 802.1Q Default VLAN Traffic If traffic that should be in the default VLAN is not passing properly, first verify that the default VLAN is set correctly, as see above. Also note the switch can be configured to either accept or deny untagged pack- ets.
Troubleshooting 802.1Q Question: What needs to be done if the native VLAN needs to be tagged when connected to an OmniSwitch 7700/7800/8800? Answer: The Gigabit/Ethernet port can be moved into a different dummy VLAN, and then the tagged service can be created for the previous native VLAN. CLI Commands are as follows: Dummy VLAN 99 created: >vlan 99 A tag service is to be created on port 99 in VLAN 1.
Troubleshooting 802.1Q Advanced Troubleshooting To verify the 802.1Q configuration from the CMM perspective use the debug 802.1q CLI command. If frame type is set to all, then the egress default VLAN will equal the ports default VLAN. If force tag internal is set to off, e.g.
Page 138
Advanced Troubleshooting output definitions Aggregate/Slot Status Port Status General Info Hardware Info page 8-6 Whether the slot or aggregate link is actively running. Whether the port is actively running. Provides general information on the modules in the chassis, including module type, number of ports, and ASIC. Lists the various debug messages for the selected slot and port.
Troubleshooting 802.1Q Dshell Commands The first step in troubleshooting for an 802.1Q problem through Dshell is to verify the configurations. Validating the configurations in Dshell remove the chances of inconsistency between the CLI and Dshell. Following is the list of commands to verify the configuration of the 802.1Q ports in Dshell. These commands will verify that there is no inconsistency between the CLI and Dshell.
Page 140
Dshell Commands 5:0 value = 23 = 0x17 5:0 nidbg> print_default_vlan_8021q_ni 23 print_default_vlan_8021q_ni: 0x17 5:0 nidbg> print_configured_list_aggregate_8021q_ni print_configured_list_8021q_ni: Aggregate number = 2, number_of_configured_ vlans = 3, vlan list = 30,40,50, 5:0 value = 2 = 0x2 5:0 nidbg> print_port_aggregation_status_8021q_ni 2 print_port_aggregation_status_8021q_ni: Port number = 2, status = NOT AGGREGABLE PORT...
9 Troubleshooting Group In order to troubleshoot a VLAN Mobility problem, a basic understanding of the technology is required. Reviewing the “Assigning Ports to VLANs” and “Defining VLAN Rules” chapters in the appropriate OmniSwitch Network Configuration Guide is highly recommended. In This Chapter “Troubleshooting a VLAN Mobility Failure”...
Troubleshooting a VLAN Mobility Failure Troubleshooting a VLAN Mobility Failure There is no systematic procedure to troubleshoot a VLAN mobility issue. This section will give you a checklist, with a generally best course of action to take to determine the source of the VLAN mobility fail- ure.
Troubleshooting Group Mobility With VLAN mobility, it is critical that the network administrator have a good understanding of the traffic on their network in order to assign proper rules. It is not only important to verify the required rules have been configured, but it is also important to understand the concept of precedence for rules that may over- lap.
Troubleshooting a VLAN Mobility Failure Precedence Due to the variety of rules that can be configured there can be traffic that can match multiple rules, i.e. an IP frame could match a network address rule as well as a protocol rule. For this reason, all rules are arranged in a precedence.
= IPX SNAP Frame = 802.3 Note. OmniSwitch 6624/6648 switches do not have the protocol CAM. Instead, database shown above is maintained in software. The debug vlan rule memory command displays the memory allocated for the group mobility rules, as shown below: ->...
Dshell Dshell Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware. NI Debug Dshell Use the gmnSetPrintDestination command from the NiDebug Dshell command to redirect the output of the group mobility commands to the current session.
Troubleshooting Group Mobility 6800 Group Mobility Troubleshooting The following commands and debugging functions are available for troubleshooting Group Mobility on the NI. A summary is shown below. show vlan rules • gmHelp • gmcKiteDebug • gmcShowPorts • gmcShowRules • gmnKiteDebug •...
6800 Group Mobility Troubleshooting *************************************************** Group Mobility Help for NI *************************************************** gmnKiteDebug = 1 ------- Enable real time debugging gmnKiteDebug = 0 ------- Disable real time debugging gmnKiteShowEframe = 1 -- Display E_FRAME_PARAM when gmnKiteDebug is enabled gmnKiteShowEframe = 0 -- Do not display E_FRAME_PARAM when gmnKiteDebug is enabled gmnKiteShowDefVlan -------- Display def vlan stored at gmn gmnKiteShowPiMap -------- Display protoCam0 and protoCam1...
10 Troubleshooting QoS In order to troubleshoot Quality of Service (QoS), a basic understanding of the concept is required. Some basic concepts are covered below. Reading the “Configuring QoS” and “Configuring ACLs” chapters in the appropriate OmniSwitch Network Configuration Guide is also highly recommended. In This Chapter “QoS Behavior”...
Default queues are created for each port on the switch at start up. The switch creates additional queues based on policy rules that match incoming flows. On the OmniSwitch 6624/6648, 4 default queues are created for each port at startup. Additional queues are not created.
Troubleshooting QoS On the OmniSwitch 6624/6648, flows always share queues. On the OmniSwitch 7700/7800/8800, flows may share queues if they match the same policy and the policy action is configured for sharing through the policy action CLI command. In order to be shared, the flows must arrive on the same slice and be destined for the same egress port.
Layer 2 and Layer 3/4 conditions should not be combined. • Layer 2 conditions cannot combine source and destination parameters. • On the OmniSwitch 6624/6648, source and destination parameters may not be combined in the same • condition. Rules Order The order of entry when defining rules use policy rule command with the precedence option.
Troubleshooting QoS Viewing QoS Settings When troubleshooting, it is essential to keep track of all your QoS settings that are effective; i.e. that have been applied. A good way to display all the QoS settings is to use the show configuration snapshot qos CLI command, which generates a snapshot file of the switch’s QoS current running configuration.
Troubleshooting QoS Validation In order to validate the policy which are not applied yet (pending policies) or you may want to see how theoretical traffic would be classified by policies that are already applied on the switch, the show policy classify CLI command can be used.
Troubleshooting QoS What happens when some traffic comes in for ip destination port 80? Since it does not satisfy condition “oktftp”, it depends on the global disposition for router and bridged traffic (qos default routed disposition and qos default bridged disposition). In our case, the global disposition is default; i.e. accept. We will receive the result below to accept the traffic when no rules are matched.
Forgetting to set a rule to be reflexive could be the cause of troubles. Ask yourself when you configure a rule, “what about the reverse flow?” Note. The OmniSwitch 6624/6648 does not support reflexive rules; you have to configure a rule for the reverse flow.
Troubleshooting QoS For example, a TFTP session in one direction will prompt a reply back from the host. If a policy is created to deny traffic from that host to the switch, the replies from the host will still be accepted on the switch if the TFTP session policy is configured as a reflexive policy.
Page 160
Troubleshooting QoS To view the QoS log, use the show qos log command. The display is similar to the following: -> show qos log **QOS Log** Validate classify: valid Conditionop noip (3) Validate condition: valid Conditionop noip (3) Validate condition: valid Conditionop noip (1) Actionop noip (3) Validate action: valid...
Troubleshooting QoS QoS Statistics The show qos statistics CLI command displays statistics about the global QoS configuration as shown below: -> show qos statistics QoS stats L3 Ingress: L3 Egress: IGMP Join: Fragments: 0 Bad Fragments: 0 Unknown Fragments: 0 Sent NI messages: 9 Received NI messages: 4322 Failed NI messages: 0...
(slice 0). On the OmniSwitch 8800, each slot may have up to 4 slices (slices 0 to 3). On the OmniSwitch 6624/6648, each block of 24 ports makes up a slice (slice 0 and slice 1). (The uplink slots are part of slice 0.)
Troubleshooting QoS OmniSwitch 6624/6648 Dshell Troubleshooting Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
Page 164
Troubleshooting QoS output definitions (continued) QVID EGR0 EGR1 subnet page 10-14 Egress VLAN ID. A value appears in this field if the packet is being routed to the destination IP address, which is the address that appears in the IP field. The egress slot/port on ASIC 0.
Troubleshooting QoS Example QoS Rules See below for the steps to create a rule for blocking an offending MAC address using the CLI on the OmniSwitch. Please note the rule does not take effect until you use the qos apply CLI command. Any time you make a change you need to reissue the qos apply command for it to take.
Page 166
Example QoS Rules Example of a traffic shaping rule: -> qos stats interval 30 log level 7 log console -> policy condition ip_traffic2 source ip 192.168.10.20 -> policy action BW maximum bandwidth -> policy rule flowShape condition ip_traffic2 action BW ->...
ARP Protocol Failure ARP Protocol Failure 00:d0:95:79:62:d1 Segment: A Workstation: A 08:00:20:a8:f0:8a If device A is not able to communicate with device B, it could be a result of ARP resolution failure. To troubleshoot ARP the first reference point is to make sure that the MAC address of device A and device B are learned on the right port and in correct VLAN.
Page 169
Troubleshooting ARP When device A ARPs for the gateway IP address exist on the switch, an associated ARP cache entry is created by the switch. This entry can be viewed by using the show arp command. To search for a specific ARP entry, use the following command syntax: show arp ip-address For example: ->...
Page 170
ARP Protocol Failure To confirm the MAC address of the routing instance, use the following command: -> show ip interface vlan 10 vlan 10 Link type Link status SNMP interface index Interface index Enable IP forwarding Administrative status Operational status Enable trap Internet address Broadcast address...
Troubleshooting ARP Common Error Conditions If the ARP is not getting resolved in either of the two workstations, then the following conditions may exist: General health of the switch or NI. • Physical link status might not be operational • MAC address not learned on the port •...
Advanced ARP Troubleshooting Advanced ARP Troubleshooting If the MAC addresses is already learned on the port and the ARP is not getting resolved then we can further troubleshoot on the switch to see if the ARP request is reaching the switch and switch is respond- ing back.
Page 173
Troubleshooting ARP Some devices may be silent like printers. They ARP at the time of the bootup but after that they do not ARP at all. In order to accommodate those devices OmniSwitch allows different choices: To increase the ARP time out value. By default the ARP timeout value is set for 300 seconds. It can be •...
Dshell Troubleshooting Dshell Troubleshooting In order to troubleshoot ARP cache make sure all the steps mentioned in the earlier sections have been taken. Dshell should be used when no more error collection can be done from the CLI and debug CLI. Note.
Page 175
Troubleshooting ARP 10.40.212.3 00:d0:95:6b:4c:e9 10.40.212.4 00:d0:95:7c:7d:78 10.40.212.127 08:00:20:b0:ea:d1 10.40.212.238 00:c0:4f:12:f7:1b 192.168.50.1 00:d0:95:82:05:16 192.168.50.2 00:d0:95:83:e7:81 192.168.50.5 00:d0:95:6a:f5:bb 192.168.51.5 00:d0:95:6a:f5:bc 192.168.52.5 00:d0:95:6a:f5:bd 192.168.53.5 00:d0:95:6a:f5:be 192.168.54.5 00:d0:95:6a:f5:bf 192.168.56.2 00:d0:95:83:e7:87 192.168.57.1 00:d0:95:82:05:1d 192.168.57.2 00:d0:95:83:e7:88 192.168.57.5 00:d0:95:6a:f5:c2 192.168.58.5 00:d0:95:6a:f5:c3 NiDebug>>>quit Source Port is shown as 481. It is calculated based on Coronado ports. Each Coronado has 32 ports. 32*16=512 ports is the total Coronado port that can exist on OS7800.
Dshell Troubleshooting Viewing the ARP Table on OmniSwitch 6624/6648 Switches To look at the ARP table on OS-6600 use the following command in Dshell: Working: [Kernel]->ipni_arpShow Slot 2. NI Arp Table destination gateway 2.2.2.100 00:00:5e:00:01:02 4.4.4.1 00:d0:95:84:07:1e 4.4.4.100 00:00:5e:00:01:04 10.255.13.2 00:20:da:0a:54:10 10.255.13.90...
In order to troubleshoot an IP Routing problem, a basic understanding of the IP protocol/feature is required. Some basic concepts are covered below. Reading RFCs 791, 1812 and 1716 are highly recom- mended to anyone implementing or troubleshooting IP Routing on their switch/network. IP Routing is a process by which layer 3 packets are forwarded between two different subnets or networks.
In This Chapter In This Chapter “Introduction” on page 12-3 “IP Routing Protocol Failure” on page 12-3 “Troubleshooting via the CLI” on page 12-3 “Troubleshooting with Debug CLI” on page 12-11 “RIP Troubleshooting” on page 12-13 “OSPF Troubleshooting” on page 12-19 “BGP Troubleshooting”...
Troubleshooting IP Routing Introduction The primary function of IP Routing is processing Layer 3 IP packets and forwarding them in between two different networks or subnets. This is broken down into two functions. First is determining the best path to get from one network or subnet to the next and second is to forwarding the packet into that destination network.
Page 180
Troubleshooting via the CLI 4/19 default 4/20 default 4/21 default 4/22 default forwarding 4/23 default If the destination VLAN gateway address does respond to a ping, there should be no issue with routing, and the cause is likely to be with Source Learning assuming all other items check out properly (i.e. PC IP setup, link status, etc.) (see Chapter 2: Managing Source Learning).
Page 181
Troubleshooting IP Routing domainName nameServer(s): Verify that the switch has a valid route to the destination subnet via the show ip route command: -> show ip route Dest Address Subnet Mask ------------------+-----------------+-----------------+---------+----------- 127.0.0.1 255.255.255.255 192.168.1.0 255.255.255.0 192.168.10.0 255.255.255.0 192.168.10.1 255.255.255.255 If a route is listed to the destination’s network, you should be able to ping it.
Page 182
Troubleshooting via the CLI Destination unreachable Time exceeded Parameter problem Source quench Redirect Echo request Echo reply Address mask request Address mask reply The show ip router database command may yield a clue, possibly telling you that an interface that is designated as a router interface is down or disabled for some reason.
Page 183
Troubleshooting IP Routing Multicast packets sent Broadcast packets received Broadcast packets sent Input errors Output errors Collisions Dropped The show ip traffic command gives switch-wide statistics for traffic, and the “No Route Discards” statis- tic should somewhat resemble the “icmp stats destination unreachable” number, in that both numbers should be increasing at a similar rate.
Page 184
Troubleshooting via the CLI output definitions Local Address Local Port Remote Address Remote Port State The show udp statistics command displays UDP errors and statistics. -> show udp statistics Total datagrams received Error datagrams received No port datagrams received = Total datagrams sent Output fields are described below: output definitions...
Page 185
Troubleshooting IP Routing The show ip config command displays IP configuration on the switch: -> show ip config IP directed-broadcast = ON, IP default TTL There are user-configurable parameters that can be changed as per requirement. The vlan mtu-ip command sets the MTU (Maximum Transmission Unit) size for a VLAN. ->...
Page 186
Troubleshooting via the CLI 10.1.99.0/24 10.1.99.0/24 10.1.99.0/24 10.11.5.0/24 10.40.100.0/24 10.40.105.0/24 10.40.108.0/24 10.40.110.0/24 page 12-10 192.168.60.2 OSPF 192.168.61.2 OSPF 192.168.62.2 OSPF 10.11.5.2 LOCAL 10.40.100.2 LOCAL 10.40.105.2 LOCAL 10.40.108.2 LOCAL 10.40.110.2 LOCAL OmniSwitch Troubleshooting Guide Troubleshooting IP Routing September 2005...
Troubleshooting IP Routing Troubleshooting with Debug CLI As always, being able to obtain a trace of the traffic via a Sniffer application will tell you the bottom line. If the packets leave the source and arrive at the destination segment properly, the issue does not lie with routing, the switch, or any intermediate device.
Page 188
Troubleshooting with Debug CLI ip-address ip-pair protocol show-broadcast show-multicast The debug ip packet command syntax starts IP debugging on NI #1 to show only broadcast packets, which will include ARPs, and then outputs them to console. For example: -> debug ip packet start board ni 1 show-broadcast on output console 1 R 1/22 00d095206408->(ffffffffffff) ARP Request 192.168.1.2->192.168.1.24 1 S CMM 00d095206408->(ffffffffffff) ARP Request 192.168.1.2->192.168.1.24 1 R 1/22 00d095206408->(ffffffffffff) ARP Request 192.168.1.2->192.168.1.24...
Troubleshooting IP Routing RIP Troubleshooting The following commands are used to troubleshoot RIP failures: show ip rip interface show ip rip redis-filter show ip rip show ip rip peer show ip rip routes show ip rip debug 7700-> ip rip debug-type ? (IP Routing &...
Page 190
RIP Troubleshooting 11.40.150.1 The interface is enabled. Verify that local interface redistribution is enabled, using the commands: ->show ip rip redist Status: ACT - Active Protocol Metric -------------+-----------+----------+------ LOCAL -> show ip rip redist-filter Control: All-Sub - All Subnets Aggreg - Aggregate Permit: Perm...
Page 191
Troubleshooting IP Routing 10.10.42.159 255.255.255.255 11.40.117.0 255.255.255.0 11.40.150.0 255.255.255.0 11.40.211.0 255.255.255.0 11.41.117.0 255.255.255.0 11.41.211.0 255.255.255.0 192.168.10.0 255.255.255.0 Notice, that route 6.0.0.0 and 8.0.0.0 appears with the natural subnet mask, even though it is configured to be class C mask. This is because RIP v1 does not advertise the mask and router always assume the natural mask.
Page 192
RIP Troubleshooting error warning recv send redist info setup time Any combination of debug-types can be set. Lets look at all the RIP debug messages by using the option all: -> ip rip debug-type all -> ip rip debug-level 255 tRip-: processRipNetQueue: Enter.
Page 193
Troubleshooting IP Routing tRip-: Adding 11.40.1.0/24->11.40.211.1 to FIB tRip-: in ripRdbLookup for 11.40.117.0 (255.255.255.0) tRip-: Adding 11.40.117.0/24->11.40.211.1 to FIB tRip-: in ripRdbLookup for 11.40.150.0 (255.255.255.0) tRip-: Adding 11.40.150.0/24->11.40.211.1 to FIB tRip-: in ripRdbLookup for 11.41.117.0 (255.255.255.0) tRip-: Adding 11.41.117.0/24->11.40.211.1 to FIB tRip-: in ripRdbLookup for 11.41.211.0 (255.255.255.0) tRip-:...
Page 194
RIP Troubleshooting tRip-: in ripRdbLookup for 10.0.0.0 (255.0.0.0) tRip-: ripSupply: gw=0.0.0.0 met=16 tRip-: ripSupply: Forcing metric for 10.10.42.57 to INFINITY (split horizon) tRip-: in ripRdbLookup for 10.0.0.0 (255.0.0.0) tRip-: ripSupply: gw=0.0.0.0 met=16 tRip-: ripSupply: Forcing metric for 10.10.42.159 to INFINITY (split horizon) tRip-: in ripRdbLookup for 10.0.0.0 (255.0.0.0) tRip-:...
Troubleshooting IP Routing OSPF Troubleshooting The following commands are used to troubleshoot OSPF failures: show ip ospf interface x.y.z. show ip ospf area 2.2.2.2 show ip ospf show ip ospf neighbor //state is FULL (connect to DR or BDR) or 2 Ways (router to router) show ip ospf interface show ip ospf lsdb A debug-level of 50 for detail and 75 for more detail.
Page 196
OSPF Troubleshooting # of Init State Neighbors # of Exchange State Neighbors # of Full State Neighbors This interface has been assigned to area 0.0.0.5. OSPF interface status is down because the administrative status of the OSPF interface is down. If the priority of the interface is set to 0 then this interface will not participate in the elections for DR and BDR.
Page 197
Troubleshooting IP Routing External LSDB Limit Exit Overflow Interval # of SPF calculations done # of Incr SPF calculations done # of Init State Nbrs # of Exchange State Nbrs # of Full State Nbrs # of attached areas # of Active areas # of Transit areas # of attached NSSAs Since, OSPF is enabled globally so enable OSPF on the interface.
Page 199
Troubleshooting IP Routing 192.168.54.0 10.26.0.1 192.168.55.0 10.26.0.1 192.168.56.0 10.26.0.1 192.168.57.0 10.26.0.1 192.168.58.0 10.26.0.1 192.168.59.0 10.26.0.1 192.168.60.0 10.26.0.1 192.168.61.0 10.26.0.1 192.168.62.0 10.26.0.1 These routes may be using RIP v1 or v2, static or local route redistribution. Therefore a separate table is maintained for all the external link states.
Page 200
OSPF Troubleshooting If local, static, or any other external protocol routes need to be redistributed into OSPF then the first step is to make that OSPF router to be a AS Border Router. This need OSPF status to be disabled. ->...
Page 201
Troubleshooting IP Routing send flood lsdb vlink redist summary dbexch hello auth area intf info setup time OmniSwitch Troubleshooting Guide Administratively enables/disables debugging messages for packets sent by OSPF only. Administratively enables/disables debugging messages for the flooding of Link State Advertisements (LSAs) in OSPF only. Administratively enables/disables debugging messages for OSPF’s Shortest Path First (SPF) calculations only.
Page 202
OSPF Troubleshooting Let’s look at all the messages that appear on the console during the setup of OSPF adjacency. The enabled debug types are state, hello and area using the command: This command is too verbose so special care should be taken when using this command. ->...
Troubleshooting IP Routing tOspf-: ospfNbrStateMachine: (10.40.110.3) Change! PREV LOADING; EVENT LOAD- DONE; NEXT FULL. (Loading of the LSAs done, spf calculations being done and the routes are getting loaded in the route table. The state moves to Full with the neighbor.) tOspf-: ospfAreaTimer:3356 ospfBuildRouterLsa(area 0.0.0.5, flags 0x5).
Page 204
BGP Troubleshooting -> show ip bgp neighbors statistics Neighbor address # of UP transitions Time of last UP transition # of DOWN transitions Time of last DOWN transition Last DOWN reason # of msgs rcvd # of Update msgs rcvd # of prefixes rcvd # of Route Refresh msgs rcvd # of Notification msgs rcvd...
Troubleshooting IP Routing Dshell Troubleshooting Advanced IP Routing The following Dshell commands are used to troubleshoot advanced IP routing. ipdbg=x This command runs on the NI of the 7700/8800. Value can be 'OR'ed (for example, 0x20001000 is 'OR'ed with IPDBG_DBG and IPDBG_WARN). The most common value is 0x20000000. The default value is 0x10000000.
Dshell Troubleshooting Advanced IP Routing Internet address: 127.0.0.1 Netmask 0xff000000 Subnetmask 0xff000000 Metric is 0 Maximum Transfer Unit size is 32768 46 packets received; 46 packets sent 0 multicast packets received 0 multicast packets sent 0 input errors; 0 output errors 0 collisions;...
Troubleshooting IP Routing 1:0 fe7 vlan180. [@0x01488690] Flags 0x1041 Internet address: 169.10.208.3 Netmask 0xfffff000 Subnetmask 0xfffff000 Ethernet Address: 00:d0:95:86:88:68 VRRP Ethernet Address: 00:00:00:00:00:00 Maximum Transfer Unit size is 1500 Arp timeout is 300. If address list pointer 1488600 Iprm_routeShow Shows the type of routes per NI. 1:0 nidbg>...
Overview Overview VRRP specifies an election protocol. All protocol messaging (not user data) is performed using IP multi- cast datagrams. The Multicast IP address is 224.0.0.18. This allows VRRP to operate over a variety of LAN technologies supporting IP. There source MAC address for these datagrams is also specified in the RFC.
Troubleshooting Virtual Router Redundancy Protocol (VRRP) Protocol Information This next section describes how VRRP routers exchange information. IP Field Descriptions Source Address Destination Address Protocol VRRP Field Descriptions Version Type VRID Priority Count IP Address Authentication Type VRRP States There are only 3 states that a VRRP Router can be in. They are initialize, Master, and Back-up. OmniSwitch Troubleshooting Guide The primary address of the interface the packet is being sent from.
OmniSwitch 7700/7800/8800 Implementation OmniSwitch 7700/7800/8800 Implementation This section will cover how and what the OmniSwitch 7700/7800/8800s will support. VRRP Security The OmniSwitch7700/7800/8800 will support no authentication and simple text password. However the third method mentioned in the RFC, IP Authentication with HD5 HMAC is not supported in this the current release of VRRP software.
Troubleshooting Virtual Router Redundancy Protocol (VRRP) CMM Failover When the CMM receives a Takeover message from the Chassis Supervisor it will first inform ARP to purge all entries for the virtual router IP/Mac addresses. VRRP will then continue with a normal start-up procedure, even though interfaces are already enabled upon bootup of secondary.
VRID Errors VRID VLAN page 13-6 Troubleshooting Virtual Router Redundancy Protocol (VRRP) The virtual router ID, in the range from 1–255 (OmniSwitch 7700, 7800, or 8800) or 1–7 (OmniSwitch 6624 or 6648). VRID Errors UpTime Become Master Adv. Rcvd 378890...
Page 217
Troubleshooting Virtual Router Redundancy Protocol (VRRP) output definitions (continued) State UpTime Become Master Adv. Rcvd -> show vrrp 1 statistics Virtual Router VRID = 1 on VLAN = 1 State UpTime (1/100th second) Become master Advertisement interval errors Password errors Authentication errors Authentication type errors IP TTL errors...
Page 218
CMM Failover output definitions (continued) Authentication type errors IP TTL errors IP address list errors Zero priority advertisements sent Zero priority advertisements received Release History Release 5.1; command was introduced. page 13-8 Troubleshooting Virtual Router Redundancy Protocol (VRRP) The total number of VRRP packets received in which the AuthType value was different than the one configured for the virtual router.
ARP Table ARP Table The ARP Table of the OmniSwitch that is the Master Router will have the Virtual MAC Learned in the ARP Table. It will not be learned on a port. In the example below I use the show arp command to illus- trate before and after a switch the VRRP master.
Troubleshooting Virtual Router Redundancy Protocol (VRRP) Dshell Troubleshooting In a situation where VRRP is flapping, go into Dshell of the switch that is configured as the backup virtual router that is flapping and enter vrrpTMon(1, vrid, vlanId). For example, for virtual router 200 on VLAN 200 the command will be: vrrpTMon(1,200,200).
14 Troubleshooting IP Multicast Switching (IPMS) In order to troubleshoot IP Multicast Switching, a basic understanding of its function is required. Some basic concepts are covered below. OmniSwitch 7700/7800/8800 supports IP Multicast Switching and Routing. Reading the “Configuring IP Multicast Switching” chapter in the appropriate OmniSwitch Network Configuration Guide is also highly recommended.
Troubleshooting IP Multicast Switching (IPMS) Troubleshooting a Device that Cannot Join an IP Multicast Stream If a device cannot join a stream, the first thing to do is to verify Layer 2/Layer 3 connectivity and that no physical errors exist. The next step is to look at the switch the device is attache to, see if the device is a member of the multicast group.
Troubleshooting IP Multicast Switching (IPMS) Troubleshooting a Device that Drops Out of an IP Multicast Stream If the issue is a device can actually join a multicast stream, but loses it after a period of time, there are a few items to check. First, does the device lose the stream when the Expiry timer reaches zero in the show ip multicast groups command? ->...
Page 226
Troubleshooting IP Multicast Switching (IPMS) Is Multicast Switching enabled on your switch? If it is not enabled, you will likely notice high utilization for the switch, and devices in the VLANs where multicast traffic is flowing will be being flooded with the stream(s).
Page 227
Troubleshooting IP Multicast Switching (IPMS) OmniSwitch Network Configuration Guide. Also, see the “IPMS Application Example” section as it gives a good example of how and when to use several of the IPMS commands. To find where a multicast stream begins in your network, you can use the show ip multicast forwarding command.
Page 228
Troubleshooting IP Multicast Switching (IPMS) You may also have a misconfigured or malfunctioning ip multicast static-querier. In the show ip multicast queriers command, look for Type of “Static” and check to see if the IP addresses configured exist and are functioning properly. Also mentioned was that if a group membership expires it may be because it isn’t on an IP VLAN.
Troubleshooting IP Multicast Switching (IPMS) Troubleshooting IPMS in Debug CLI The Debug CLI will allow you to view traffic traversing the switch in many ways. The most useful command for troubleshooting IP Multicast is: -> debug ip packet show-multicast on board ni 1 output console This command turns on debug for IP packets, turns on the ability to show multicast packets, looks only on blade #1, and outputs to console.
Page 230
Troubleshooting IP Multicast Switching (IPMS) Note the output of the show ip multicast queriers command during this test: -> show ip multicast queriers Source IP VLAN Slot/Port Expire --------------------+----+---------+------+------- 10.10.10.190 The .190 address is that of the uplinked OSR9 in the test. The show ip multicast neighbors command will show the neighbors in this test network: ->...
Troubleshooting IP Multicast Switching (IPMS) Dshell Troubleshooting The IPMS application has its own shell to verify the specific data displayed on the CLI. Use a question mark to display the local list of commands for each level. Every CLI commend has a corresponding output on this shell with extended information on each entry.
Page 232
Troubleshooting IP Multicast Switching (IPMS) grp - Group Membership nbr - Neighbors qry - Queriers src - Sources prx - Proxy vprx - Proxy by Vlan v3prx - IGMPv3 Proxies qint - Querier interface list qtmr - Querier timer list hwrt - Toggle hardware flag enbl - Toggle enable flag IPMEM>...
15 Troubleshooting DVMRP In This Chapter “Introduction” on page 15-2 “DVMRP Troubleshooting” on page 15-2 “DVMRP Global and Interface Commands” on page 15-2 “DVMRP Debug Commands” on page 15-4 OmniSwitch Troubleshooting Guide September 2005 page 15-1...
Introduction Introduction DVMRP is used to route Multicast packets through different IP Networks. This is a Dense Mode Multi- cast Routing Protocol. Dense Mode refers to the network environment the protocol was designed to service. Dense Mode protocols are designed for LAN environments where there are a lot of users and bandwidth is plentiful.
Page 245
Troubleshooting DVMRP Route Timeout Number of Routes Number of Reachable Routes = 6 -> show ip dvmrp interface Address Vlan -----------------+------+--------+--------+---------------+------------- 192.168.10.1 192.168.11.1 192.168.12.1 If an interface is not enabled then no multicast traffic will be routed to that VLAN the Interface represents. What if everything is enabled in this switch correctly? It could be another switches problem or this switch may be unable to process the messages correctly.
DVMRP Troubleshooting 192.168.10.1 192.168.11.1 192.168.12.1 Remote routes will be seen if they are learned from another switch. Again be sure that the interfaces are enabled. If that is the case it possible that the other switch is not configured correctly. If the above commands look good in your switch and in the other switches you can use the following command to see if the switch has a next hop.
Page 247
Troubleshooting DVMRP The third command turns on the debug level. The low is 0, which is no debugging at all, and the high is 110, which prints everything to the screen. There is so much going on in the switch and so much put to the screen that an explanation of what level would bring what output.
Page 248
DVMRP Troubleshooting tDvmrp-: dvmrpRibRemoveHoldDown: 192.168.11.0-255.255.255.0 tDvmrp-: dvmrpRibDelinkHoldDownQ: 192.168.11.0-255.255.255.0 tDvmrp-: dvmrpInitChildAndSubs: All V1 nbrs dependent on us for rt 192.168.11.0 tDvmrp-: dvmrpDeleteGListEntry: rt 192.168.11.0-255.255.255.0 tDvmrp-: dvmrpEnable: V3 Config=1 Oper=2 tDvmrp-: dvmrpProtoEnabledOnVlan: V3 mprotos=0x0 tDvmrp-: dvmrpSendIpmrmProto: V3 Configured tDvmrp-: MIP and TM says we’re enabled. tDvmrp-: dvmrpAddMembership: V3 do IP_ADD_MEMBERSHIP tDvmrp-:...
Page 249
Troubleshooting DVMRP tDvmrp-: #subords=0, pruncnt=0 tDvmrp-: Looking at V1... tDvmrp-: V1 not sub/nbr tDvmrp-: Looking at V2... tDvmrp-: V2 not sub/nbr tDvmrp-: V2 not rib->upstrVl=1, may need forwarding tDvmrp-: Group 224.2.142.227 not learned on V2 tDvmrp-: Looking at V3... tDvmrp-: V3 not sub/nbr tDvmrp-: V3 not rib->upstrVl=1, may need forwarding...
Page 250
DVMRP Troubleshooting tDvmrp-: IGMP packet from 192.168.10.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-: IGMP packet from 192.168.12.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-: IGMP packet from 192.168.12.1 tDvmrp-: IGMP packet from 192.168.12.1 tDvmrp-: dvmrpRecvReport: tDvmrp-: On V3 Unable to peer with nbr 192.168.12.3 tDvmrp-: dvmrpAddNeighbor: 192.168.12.3 new nbr...
Page 251
Troubleshooting DVMRP tDvmrp-: IGMP packet from 192.168.10.1 tDvmrp-: IGMP packet from 192.168.10.1 tDvmrp-: IGMP packet from 192.168.10.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-: IGMP packet from 192.168.12.1 tDvmrp-: IGMP packet from 192.168.12.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-: IGMP packet from 192.168.11.1 tDvmrp-:...
Page 252
DVMRP Troubleshooting tDvmrp-: Found S,G matching source network 255.255.255.0 tDvmrp-: Creating a new prune state S,G 192.168.10.25 224.2.142.227 tDvmrp-: tDvmrp-: dvmrpPruneTimeEnQ: tDvmrp-: dvmrpComputeForwardingVector: tDvmrp-: For rt 192.168.10.0 - grp 224.2.142.227 tDvmrp-: #subords=1, pruncnt=1 tDvmrp-: Looking at V1... tDvmrp-: Looking at V2... tDvmrp-: V2 not rib->upstrVl=1, may need forwarding tDvmrp-:...
Page 253
Troubleshooting DVMRP tDvmrp-: V3 is a sub/nbr, numFwdIfs=2 tDvmrp-: V3 not rib->upstrVl=1, may need forwarding tDvmrp-: Group 224.2.178.69 not learned on V3 tDvmrp-: # of IFs to fwd to: 2 tDvmrp-: Forwarding vector list changed, return 1 tDvmrp-: dvmrpComputeForwardingVectorForRoute: Update IPMRM tDvmrp-: dvmrpSendIpmrmRoute: Update/add to IPMRM S,G tDvmrp-:...
Page 254
DVMRP Troubleshooting tDvmrp-: Metric orig=32, adj=32 tDvmrp-: dvmrpUpdateUnreachable: tDvmrp-: Nbr[192.168.11.2] no longer a subordinate for rt 192.168.10.0 tDvmrp-: dvmrpUpdateUnreachable Nbr[192.168.11.2] no longer a subordi- nate for rt 192.168.10.0 tDvmrp-: dvmrpComputeForwardingVectorForRoute: call it... tDvmrp-: dvmrpComputeForwardingVector: tDvmrp-: For rt 192.168.10.0 - grp 224.2.142.227 tDvmrp-: #subords=1, pruncnt=1 tDvmrp-:...
Page 256
DVMRP Troubleshooting tDvmrp-: #subords=0, pruncnt=0 tDvmrp-: Looking at V1... tDvmrp-: V1 not sub/nbr tDvmrp-: V1 not rib->upstrVl=2, may need forwarding tDvmrp-: Group 224.2.178.69 not learned on V1 tDvmrp-: Looking at V2... tDvmrp-: V2 not sub/nbr tDvmrp-: Looking at V3... tDvmrp-: V3 not sub/nbr tDvmrp-: V3 not rib->upstrVl=2, may need forwarding...
Page 257
Troubleshooting DVMRP tDvmrp-: dvmrpUpdatePoisoned: nbr 192.168.12.3 is dependent on us for rt 192.168.13.0 tDvmrp-: dvmrpComputeForwardingVectorForRoute: call it... tDvmrp-: dvmrpComputeForwardingVector: tDvmrp-: For rt 192.168.13.0 - grp 224.0.1.24 tDvmrp-: #subords=1, pruncnt=0 tDvmrp-: Looking at V1... tDvmrp-: V1 not sub/nbr tDvmrp-: V1 not rib->upstrVl=2, may need forwarding tDvmrp-: Group 224.0.1.24 not learned on V1 tDvmrp-:...
Page 258
DVMRP Troubleshooting tDvmrp-: Group 224.2.178.69 not learned on V1 tDvmrp-: Looking at V2... tDvmrp-: V2 not sub/nbr tDvmrp-: Looking at V3... tDvmrp-: V3 is a sub/nbr, numFwdIfs=1 tDvmrp-: V3 not rib->upstrVl=2, may need forwarding tDvmrp-: Group 224.2.178.69 not learned on V3 tDvmrp-: # of IFs to fwd to: 1 tDvmrp-:...
Page 259
Troubleshooting DVMRP -> tDvmrp-: dvmrpRecvIpmrmDelEntry: V2 tDvmrp-: tDvmrp-: Found route 192.168.13.0 to ipsa tDvmrp-: S,G entry found for deletion tDvmrp-: dvmrpPurgeGroup: tDvmrp-: dvmrpDelPrnSentForGrp: tDvmrp-: dvmrpRecvIpmrmDelEntry: V2 tDvmrp-: tDvmrp-: Found route 192.168.13.0 to ipsa tDvmrp-: S,G entry found for deletion tDvmrp-: dvmrpPurgeGroup: tDvmrp-: dvmrpDelPrnSentForGrp:...
Page 260
DVMRP Troubleshooting tDvm-> rpshow -: tDvmrp-: dvmrpUpdateReachable: tDvmrp-: dvmrpRibResetAgeoutTimer: -> show ip dvmrp route tDvmrp-: dvmrpRecvReport: tDvmrp-: dvmrpUpdateRoute: V2 UPDATE route for 192.168.10.0- 255.255.255.0 tDvmrp-: tDvmrp-: dvmrpUpdatePresentRoute: tDvmrp-: Metric orig=34, adj=34 tDvmrp-: dvmrpUpdatePoisoned: tDvmrp-: Received on diff vlan tDvmrp-: Nbr 192.168.11.2 indicating dependency for [192.168.10.0- 255.255.255.0] tDvmrp-: dvmrpUpdatePoisoned: nbr 192.168.11.2 is dependent on us for rt...
16 Troubleshooting PIM-SM In order to troubleshoot issues related to Protocol Independent Multicast-Sparse Mode (PIM-SM), a basic understanding of the protocol—as well as IP multicast technology—are required. Basic PIM-SM concepts are explained below; for detailed protocol specifications please refer to RFC 2362 (PIM-SM) as well as the OmniSwitch 7700/7800/8800 Advanced Routing Configuration Guide, which contains a protocol over- view and PIM-SM configuration information.
Introduction Introduction Traditional multicast routing protocols like DVMRP, MOSPF or PIM-DM were implemented to provide multicast routing in campus network. These traditional dense mode multicast protocol were intended for use within regions where a group is widely represented or bandwidth is not an issue. However when group members and senders to these groups are sparsely distributed across a wide area, traditional multicast rout- ing protocol schemes do not provide an efficient way to establish distribution trees.
Troubleshooting PIM-SM Protocol Overview DR Election Hello messages are sent periodically between PIM neighbors. This informs routers which interface has PIM neighbors. Hello messages are multicast packets using address 224.0.0.13, which corresponds to ALL-PIM-ROUTERS group. When a router receives a Hello message, it stores the IP address for that neighbor and determines the Designated Router (DR) for that specific interface.
Protocol Overview PIM: ----- PIM Header ----- PIM: PIM: Version PIM: Message Type PIM: Reserved PIM: Checksum PIM: Option Type PIM: Option Length PIM: Option Value Debugging Hello Messages To debug DR election and view Hello messages sent and received on PIM router interface, use the follow- ing commands: ->...
Troubleshooting PIM-SM Related CLI Command To view if a PIM router interface is the DR for the LAN segment, enter the following command: -> show ip pimsm neighbor Neighbor Address Vlan -----------------+--------+-----------+-----------+--------- 192.168.11.2 192.168.12.2 If (DR) is not present it means the interface has not DR role on the segment. OmniSwitch Troubleshooting Guide Uptime Expires...
BSR Election BSR Election Candidates-BSR sends Bootstrap messages within its PIM-SM domain. Bootstrap messages are multicast to the ALL-PIM-ROUTERS group. Bootstrap message parameters C-BSR ID, which are equal to the BSR IP address, and the C-BSR priority, are used for the BSR election. The BSR will be the router with the highest priority;...
Troubleshooting PIM-SM Simplified Packet Format IP: ----- IP Header ----- IP: Protocol IP: Source address IP: Destination address = [224.0.0.13] PIM: ----- PIM Header ----- PIM: PIM: Version PIM: Message Type PIM: Reserved PIM: Checksum PIM: Fragment tag PIM: Hash mask length PIM: BSR-priority Debugging BSR/Bootstrap Commands to debug received and forwarded bootstrap, the command will return information on BSR...
BSR Election The router selects itself as initial BSR. tPimsm-: Received bootstrap message from 192.168.11.1, bsr addr:192.168.11.1 on vlan 1 tPimsm-: Ignoring less preferred bsr 192.168.11.1, Pri 0. Bsr 192.168.11.2, Pri 0 Bootstrap message received and action taken, notice BSR ID and BSR priority. tPimsm-: Received bootstrap message from 192.168.11.1, bsr addr:192.168.11.1 on vlan 1...
Troubleshooting PIM-SM Related CLI Command To view which routers are assuming the role of the BSR, expiry time, C-BSR address, and C-BSR prior- ity, type: -> show ip pimsm Status BSR Address BSR Expiry Time CBSR Address CBSR Mask Length CBSR Priority IF you don’t want a PIM router to assume BSR role, enter the following syntax: ->...
C-RP Advertisements C-RP Advertisements Candidate-RPs advertisements are periodically unicast from the C-RP to the BSR. These advertisements contain group multicast address the router can be responsible for and the priority for the corresponding group address. C-RP could be configured to participate as RP for specific multicast groups or for all multi- cast groups.
C-RP Advertisements Another output from a C-RP advertisement, advertises itself for all multicast groups. All group addresses are represented by the pair 224.0.0.0 240.0.0.0, which covers 224.0.0.0 up to 239.255.255.255. tPimsm-: Recv. CRP-Adv (RP:192.168.10.3,Prefix Cnt:1, Pri:0) from 192.168.10.3:192.168.11.1 on vlan 2 tPimsm-: RP : 192.168.10.3 : Prefix: 224.0.0.0 Mask : 240.0.0.0 Related CLI Command...
Troubleshooting PIM-SM RP-SET An RP-SET contains a set of Candidates-RP IP addresses that want to participate as RP for multicast group. The RP-SET is derived from the C RP-Advertisements received by the BSR. RP-SETs are adver- tised by the BSR in a bootstrap message to all PIM SM routers by using the ALL-PIM-ROUTER address 224.0.0.13.
RP-SET PIM: *** Encoded-Unicast RP Address # 1 *** PIM: Address family = 1 (IP (IP version 4)) PIM: Encoding type = 0 PIM: Unicast address = [192.168.12.2] PIM: RP1-Holdtime PIM: RP1-Priority PIM: Reserved The RP-Holdtime parameters is the corresponding time the BSR will hold related RP multicast group information in its table as valid.
Troubleshooting PIM-SM Related CLI Command To View RP-SET on a router, use the show ip pimsm rp-set command. For example: -> show ip pimsm rp-set Group Address Address ------------------+---------------+--------+----------- 225.1.1.1/32 192.168.11.2 226.1.1.1/32 192.168.12.2 227.1.1.1/32 192.168.12.2 OmniSwitch Troubleshooting Guide Holdtime Expires 00h:00m:00s 00h:00m:00s 00h:00m:00s...
Join/Prune Join/Prune Join/Prune messages are sent by the DR to join or prune a branch off the multicast distribution tree in order to receive multicast group on a specific LAN segment that has active group members. Registration of the members is achieved with IGMP host membership report. Upon reception of such message each upstream router between the receiver and the RP creates or updates its multicast route entry for specific multicast group(s) adding interface where join/prune request have been received.
Page 281
Troubleshooting PIM-SM PIM: ----- PIM Header ----- PIM: PIM: Version PIM: Message Type = 3(Join/Prune) PIM: Reserved PIM: Checksum = 5794 (correct) PIM: PIM: *** Encoded-Unicast-Upstream Neighbor Address *** PIM: Address family = 1 (IP (IP version 4)) PIM: Encoding type = 0 PIM: Unicast address = [192.168.12.1] PIM: Reserved PIM: Number of groups...
Join/Prune PIM: ..1. = WC bit - join / prune applies to the (*,G) or (*,*,RP) entry PIM: ..1 = RPT-bit - information about (S,G) is sent towards the RP PIM: Mask length = 32 PIM: Source address = [192.168.11.2] Simplified PRUNE Packet The main difference between a JOIN and PRUNE is the number of joined source versus pruned source: PIM: *** Encoded-Multicast Group Address-1 ***...
Troubleshooting PIM-SM Register When a source starts transmitting to a multicast group, the DR on the segment encapsulates the data and sends it as unicast to the RP representing the group. The source IP address will be the DR and the destina- tion IP address will be the RP.
Troubleshooting PIM-SM Shared Tree A shared distribution tree is formed around the RP, from which all traffic is distributed regardless of the location of the traffic sources. The advantage of shared distribution trees is simple topology on PIM SM routers. The DR is sending a JOIN message to the RP, and a graft to the SPT. The disadvantage is that the path between the source and receivers might not be the shortest one, which could introduce delay.
Shared Tree Related CLI Command In order to view the IP multicast routing table as well as source and distribution tree type: -> show ip pimsm mroute Group Address Src Address ---------------+------------------+------+-----------+------+----- 225.1.1.1 192.168.14.115/32 page 16-24 Assert Assert Assert Flags Metric expires Pref 00h:00m:00s 0...
Troubleshooting PIM-SM Source-Based Tree PIM SM protocol allows a DR to build a Shortest Path Tree, which could provide a shortest path toward the source. It is being referenced as Source Based Tree because the DR closest to the receiver is initiating the process.
Source-Based Tree Related CLI Command To view the distribution tree issue: -> show ip pimsm mroute Group Address Src Address ---------------+------------------+------+-----------+------+----- 225.1.1.1 192.168.14.115/32 page 16-26 Assert Assert Assert Flags Metric expires Pref 00h:00m:00s 0 OmniSwitch Troubleshooting Guide Troubleshooting PIM-SM September 2005...
Troubleshooting Examples: Limitations -> show ip pimsm Status BSR Address BSR Expiry Time CBSR Address CBSR Mask Length CBSR Priority CRP Address CRP Hold Time CRP Expiry Time CRP Interval CRP Priority Data Timeout Join/Prune Interval Max RPs Probe Time Register Checksum Register Suppress Timeout ->...
Page 291
Troubleshooting PIM-SM -> debug ip pimsm rp Group Address RP Address ------------------+------------------ 224.0.1.22 172.100.1.254 224.0.1.24 172.100.1.254 239.0.0.90 172.100.1.254 239.1.1.200 172.100.1.254 239.1.1.201 172.100.1.254 239.1.1.202 172.100.1.254 239.255.255.254 172.100.1.254 OmniSwitch Troubleshooting Guide September 2005 Troubleshooting Examples: Limitations page 16-29...
17 Troubleshooting Server In order to successfully troubleshoot the Alcatel Server Load Balancing feature, a brief understanding of this services functions are necessary. Our Basic Definition The “Server Load Balancing” (SLB) term used in this document refers to the functionality of distributing client requests across servers logically grouped in “clusters.”...
Introduction Introduction The primary function of a Server Load Balance cluster is to provide a method to logically treat a group of physical servers (known as a server farm) as one large virtual server (known as an SLB cluster). Note. This document does not discuss the basic operation of Server Load Balancing. To learn about how SLB works, refer to “Configuring Server Load Balancing”...
Troubleshooting Server Load Balancing Troubleshooting Commands Below is a list of commands, which will be used and discussed in this chapter. show ip slb (Displays SLB information) show ip slb cluster-name For example: -> show ip slb cluster Intranet Cluster Intranet Admin status Operational status Routed flows success ratio (%) = 100,...
Troubleshooting a Complete Failure Troubleshooting a Complete Failure Gathering pertinent information is essential in order to properly characterize the problem. Obtain symp- tom facts, review all recent network or architecture changes, localize the problem, e.g. does it affect only certain floors, or departments? Devise an action plan. The first step in any troubleshooting process is to gather information.
Troubleshooting Server Load Balancing Troubleshooting a Partial Failure The number one cause of a partial failure is misconfiguration. In the following area, we will show you how to determine the SLB configuration. The Troubleshooting Procedure If you have reason to believe that the SLB cluster is operational, however inaccessible, Alcatel’s 7700/ 7800/8800 provides useful commands for narrowing down the problem.
Page 298
The Troubleshooting Procedure Troubleshooting Server Load Balancing page 17-6 OmniSwitch Troubleshooting Guide September 2005...
Authenticated VLANs In order to troubleshoot Authenticated VLAN (AVLAN), a basic understanding of how authentication works in the switch is required. Understanding of Radius and DHCP server will be useful in troubleshoot- ing Authenticated VLAN. Note. Reading the “Managing Authentication Servers” and the “Configuring Authenticated VLANs” chapters in the appropriate OmniSwitch Network Configuration Guide is highly recommended.
Troubleshooting AVLAN Troubleshooting AVLAN DHCP Request Failure If the client (PC-1) is configured to get the DHCP IP address and can not get DHCP address during the first phase of authentication process, it could be because of wrong configuration in the switch, communi- cation failure or miss configured DHCP server.
Troubleshooting Authenticated VLANs 192.168.10.100 00:c0:4f:0c:3a:e4 There are couples of other things you can verify is on the DHCP server itself. Router IP address is set to the IP address of one of the authenticated VLANs in the switch. • Address pool must be created in order to assign the DHCP IP address. •...
Troubleshooting AVLAN -> show mac-address-table Legend: Mac Address: * = address not valid Vlan Mac Address ------+-------------------+--------------+-----------+------------+----------- 00:c0:4f:0c:3a:e4 Total number of Valid MAC addresses above = 1 Now verify that the Gateway defined in RADIUS server point towards the right IP address. ARP table confirms that the switch has learned the ARP entry of the RADIUS server.
Troubleshooting Authenticated VLANs Useful Notes on Client Issues If using telnet authentication manual IP release and renew required getting the new IP after authentica- • tion. Troubleshooting Using Debug Systrace Telnet Authentication and De-authentication Do not change the aaaDebugFunction flag under Dshell, keep it the default setting. In addition with that use below commands to capture maximum debug information: ->...
Troubleshooting Using Debug Systrace Initiate the Telnet Authentication -> 2394037098 AAA 2394037225 AAA debug1 Message succefully sent 2394037384 AAA debug3 Entering in aaa_ProcessHdlMsg 2394037565 AAA debug1 AVLAN begin Evt : Req,St : IDLE, name : 0x60008 00.00.00.00.00.00 2394037621 AAA debug3 aaaAvlanUserReq ses rec 0x3ef, Avlan ses 0x60008 2394037675 AAA debug1 Ctx admin removed from No link, set in MAC link 2394037721 AAA...
Troubleshooting Authenticated VLANs Chal_Rsdebug3p link, set in 3971075352397129005 :entering in aaaAvla IDLE, name : , ses nSendOneCtxToSeconda: OryCmm K 2397304273 AAA debug2 SL 0xa0070 00.90.27.75.dc.a2, VL 103, proto 1 p 1 2397391330 AAA debug3 Entering in aaaAvlanReplyHdlMgt 2397455810 AAA debug1 Send Authent. Success, name : admin, hdl ses 0x3efx Avlan ses 0x60008 2397544001 AAA debug2 Ask stat user admin ref 0x516bcf0 00.90.27.75.dc.a2...
Troubleshooting Authenticated VLANs name : admin, Rec(TRUNCATED) 2165965890 AAA debug3 Entering in aaa_AuthRspOK 2166030420 AAA debug3 Entering in aaa_ReplyHdlMgt 2166095975 AAA debug1 Send auth Success, session 9000b 2166161550 AAA debug32166161654 ReturnST_IDLEde2166227153bug1 debug1WAIT RESP, nam e : admin Avlan se(AAA_Serv>> Ctx adminTRUNCATED) removed from Chal_Rsp link, 2166400173set in No ng in aaaAvlanAut hRspOKdebug1 21665End Authent Evt St :08777 IDLE, name : , ses :...
Troubleshooting Authenticated VLANs 1595408715 AAA info 1595473430 AAA info try=1 1595582865 AAA info try=1 1595670050 AAA debug3 [RAD] radProcPkt: Got a reply from Radius server , id = 9, code 2, length 59 1595778543 AAA info 1595865571 AAA info rcvd. 1595953731 AAA info 1596040797 AAA...
Page 312
Dshell Troubleshooting Authenticated MAC address XCAP 802.3 SNAP header Authenticated DNS name 802.1x EAPol SNAP header 802.1x Authentication control 802.1x Authentication share 802.1x PAE group MAC address 802.1x node MAC address Debug level Debug frame type filtered Debug user port filtered Debug dump value = 1031 = 0x407 Run the adDebugShowCounter command to see any possible errors.
Page 313
Troubleshooting Authenticated VLANs XCAP: Unexpected message identifier rcv.= 0 value = 1866 = 0x74a To verify the Authentication IP bound to each VLAN with subnet mask please run the adDebugShowAv- lanIp command. Working: [Kernel]->adDebugShowAvlanIp AD show AVLAN IP addresses VLAN number IP address Mask Authentication IP address = 10.0.0.253...
19 Troubleshooting 802.1X The 802.1X standard defines port-based network access controls, and provides the structure for authenti- cating physical devices attached to a LAN. It uses the Extensible Authentication Protocol (EAP). Note. See the “Configuring 802.1X” chapter in the appropriate OmniSwitch Network Configuration Guide for a detailed explanation about different 802.1X components.
Troubleshooting with the CLI Troubleshooting with the CLI Make sure the Radius and Accounting ports are configured the same on both switch and Radius Server. The default on the Radius Server can be either 1645/1812 for Radius and 1646/1813 for the Accounting. Layer-2: show aaa server Server name = rad1 Server type...
Page 317
Troubleshooting 802.1X Authentication : enabled, IP Router Port : none, IPX Router Port : none, Verify the status of the 802.1x port using the show 802.1x command. Read the OmniSwitch CLI Refer- ence Guide to understand the explanation for each field. Layer-2: show 802.1x 2/3 802.1x slot/port = 2/3 authenticator PAE state...
Troubleshooting Using Debug CLI Troubleshooting Using Debug CLI Assuming Radius communication takes place on UDP port 1645: Layer-2: debug ip packet protocol udp port 1645 start C S 1/1 00d09579640e->00d0956af558 IP 10.1.1.1->133.2.253.1 UDP 1025,1645 C R 1/1 00d0956af558->00d09579640e IP 133.2.253.1->10.1.1.1 UDP 1645,1025 C S 1/1 00d09579640e->00d0956af558 IP 10.1.1.1->133.2.253.1 UDP 1025,1645 C R 1/1 00d0956af558->00d09579640e IP 133.2.253.1->10.1.1.1 UDP 1645,1025 1 R CMM (00d09579640e)->(00d0956af558) IP 10.1.1.1->133.2.253.1 UDP 1025,1645...
Page 319
Troubleshooting 802.1X 440814250 AAA info 440874069 AAA info 440933046 AAA info 0xa010101 441012638 AAA info 441091272 AAA info 441152216 AAA info 441230837 AAA info 441311563 AAA info 441391033 AAA info 441451000 AAA info access-request 441550316 AAA info 441610744 AAA info try=1 441709499 AAA info...
Page 320
Troubleshooting Using Debug CLI 444336676 AAA info 444417261 AAA info 444496871 AAA info 444556834 AAA info 444636441 AAA info 444716065 AAA info access-request 444795713 AAA info 444875605 AAA info try=1 444954886 AAA info try=1 445054248 AAA debug3 [RAD] radProcPkt: Got a reply from Radius server , id = 20, code 2, length 113 445154438 AAA info...
Troubleshooting 802.1X Dshell Troubleshooting Note. Dshell commands should only be used by Alcatel personnel or under the direction of Alcatel. Misuse or failure to follow procedures that use Dshell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware. Launch the NiDebugger.
Page 322
Dshell Troubleshooting Direction Source MAC addr. = 00.00.00.00.00.00 2:0 value = 0 = 0x0 To verify the sequence of the packet flow, set the below Dshell flags to troubleshoot the issue. The best way to troubleshoot is to compare the failed case with good case. 2:0 nidbg>...
Page 323
Troubleshooting 802.1X 2:0 AD INFO-> EAPol pkt.sent out: Slot=2,Glb.port=0x22,User port=3,Frame=0x4479000 2:0 AD INFO-> EAPol pkt.from supplicant, rcv.on 802.1x glb.port=0x22,user port=3: sent to 802.1x CMM: Slot=2,802.1x port's state=6,Auth.Ctrl=1,Auth.Share=2, Src.MAC addr.=00.0F.1F.D5.54.95,Cond.codes=0x8080,Frame=0x447c800 2:0 AD INFO-> EAPol pkt.sent to AAA's 802.1x: Slot=2,Glb.port=0x22,User port=3,Msg.=0x15cfe90 2:0 AD INFO-> 802.1x port config.rcv.: Msg.=0x4484000 2:0 AD INFO->...
A OS6600/OS7700/OS8800 Architecture Overview The switch benefits from an intelligent, multi-layer switching, fully distributed and passive backplane architectural design that provides redundancy of critical hardware and software elements for a continuous (non-stop) traffic processing in any network conditions without a single point of failure. Switch process- ing scheme includes a non-blocking store-and-forward crossbar design switching fabric with a distributed processing.
The MAC ASIC The MAC ASIC There are two different types of MAC layer ASICs: Catalina • Firenze • Catalina The Catalina basically provides three functions: Media Access Control (MAC) Layer functions, data buff- ering, and statistics accumulation and storage for each port. The Catalina provides the interface between Ethernet analog devices (10/100 Mbps and 1000 Mbps) and the Coronado.
Page 327
Catalina 0 1 Gig Port F-BUS Coronado 0 XY-BUS Catalina 0 Catalina 1 Gigabit port 1 Gigabit port 2 OmniSwitch Troubleshooting Guide F-BUS Coronado XY-BUS Catalina 1 1 Gig port … Catalina 6 Gigabit port 7 GNI-U/C-8 OS-8800 September 2005 The MAC ASIC GNI-U-2 OS-7XXX...
The MAC ASIC Firenze The Firenze basically provides three functions: Media Access Control (MAC) Layer functions, Data buff- ering, Flow Control and statistics accumulation and storage for each port. Firenze handles up to six Ethernet interfaces that can support independently either 10 or 100 or 1000 Mbit/ s throughput using independently either TBI or GMII interfaces.
F-BUS Coronado 0 XY-BUS Firenze 1 Firenze 0 6 Gigabit ports 6 Gigabit ports The Coronado ASIC The Coronado ASIC is the centerpiece of this advanced architectural design. While each ASIC performs a vital function in the overall architecture, the Coronado provides most of the key features like wire-rate L2 and L3 switching and routing.
The MAC ASIC The Coronado ASIC contains both Ingress and Egress functions. Ethernet frames flow from the Catalina through the Ingress Coronado, through the Nantucket switch fabric, then through the Coronado Egress logic and finally out the Egress Catalina. Note: Frames always flow through the Nantucket, even if the input and output ports are on the same Coro- nado ASIC.
Coronado: The “Brain” of the System Provides hardware performance for most features on the OmniSwitch. • Involved in both the ingress and egress packet processing. • Provides a high speed connection to Catalina via the XYBUS. XYBUS is two unidirectional busses •...
Queue Driver Interaction Queues are maintained by the software module known as the Queue Driver runs on each NI. It provides • the interaction to other software modules in the Falcon/Eagle product to create/modify/delete/manage all the queues in the system. This module provides debugging information required for the queues and maintains the statistics.
NI Supervision Queue driver sends the task initialization and task ready messages to the NI Supervision task just before receiving any configuration information from the CMM Ethernet driver. This indicates to the NI supervi- sion task that the queue driver task is ready. Source Learning L2 Destination Address Manager is a software block of Source Learning.
Queue Driver Interaction QoS Policy Deleted When a QoS Policy is deleted, the QoS Manager is notified. It checks to see if there are any flows associ- ated with the QoS Policy. The queue id for all the pseudo cam entries associated with the policy should be reprogrammed with a default queue and the existing queue has to be freed.
Link Aggregation Link Aggregation feature allows software to aggregate a set of ports (not necessarily contiguous or on the same Coronado) into a group. Each port is referred to as a channel that can carry some bandwidth. This feature requires that all the queues associated with all the ports in the group to be contiguous. Events resulting in Link Aggregation and Q driver interaction may be for the creation and deletion of link aggre- gation group.
Source Learning Source Learning Coronado ASIC is a ingress classifier that performs Layer 2 hardware table lookups to the VLAN Id corre- sponds to the incoming packets. In principle, Coronado will perform a Layer 2 Source Address (L2 SA) lookup based on whether the incoming port is a tagging or non-tagging port, a copy of the packet is re-queued to the Source Learning queue for processing if the lookup fails.
Hardware Routing Engine (HRE) This feature is responsible for managing the Coronado HRE resources that perform IP and IPX packet classification and forwarding. This functionality employees five principal Coronado resources—header cache entries, pseudo-CAM entries, hash function registers, modes, and router MACs. Header Cache Entries Contain the information necessary to modify and forward a packet, including both modifications to the packet’s content and Coronado-specific handling information.
Page 338
Hardware Routing Engine (HRE) Router MACs Helps identify candidate traffic for classification. The HRE only operates on traffic that is identified as requiring layers 3 and 4 classification. Candidate traffic for such classification must be of the type IP or IPX and must have its layer 2 destination MAC be a router MAC for this switch or be a bridged MAC that has been marked for layer 3 and 4 classification or be an IP multicast MAC.
QoS/Policy Manager Policies describe subsets of traffic, and what to do with that traffic. The Policy Processor determines what policies are enforceable, and organizes the policies into lists used by the classifier. QoS uses the same policies for Prioritization/Shaping, IP Filtering (ACLs), NAT, and IPMS Filtering. Using the same poli- cies for all these functions has several benefits: All policies affecting the traffic are centrally located •...
The Fabric Architecture The Fabric Architecture ASIC provides the switching fabric functionality for the OmniSwitch Series. The switching fabric does no frame processing and does not distinguish between L2 switching and L3 routing. The fabric provides only a limited amount of FIFO buffering for each port, flood and the multicast queue. Most of the system buff- ering is in the Ingress Coronado’s Queue Manager.
Nantucket ASIC The Nantucket ASICs have the following: Support for 8 and 16 Coronados only. • No dynamic queue numbering for each NI slot. • Fixed eight Nantuckets per Fabric board. • Maximum of one redundant Fabric board. • The Nantucket software is arranged as: Nantucket operational software resides on the CMM.
Nantucket ASIC Functional Description: The Nantucket software resides on Chassis Management Module (CMM) and run on the UltraSPARC IIe processor within the CMM. The Nantucket software communicates to the Nantucket ASIC via the PCI Bridge and Bbus Bridges. The Nantucket Software has interfaces to the following: Primary CMM SDRAM, EEPROM via the UltraSparc IIe PCI Bridge •...
Calendar Manager Module Generates dequeue requests to the Queue Manager module with priorities of uP packets highest, multi- • cast packets second highest and unicast packets lowest. Request dequeues from 512 unicast queues by sending a QID. • Request dequeues from uP queue by sending FIFO ID. •...
Page 345
Coronado Coronado Coronado Coronado Coronado Coronado Coronado Coronado Every Coronado is connected to Primary CMM and secondary CMM with all the Nantuckets. Maximum of 16 connections to Coronado can exist on each Coronado. OmniSwitch Troubleshooting Guide Nantucket 0 Nantucket 0 Nantucket 3 Secondary CMM Nantucket 1...
Roma Roma Each Fabric Module is based on two Roma ASICs, which provide the store-and-forward switching fabric function for the OS-8800 product. The Roma ASIC is bit-sliced; a group of eight Roma ASICs operates together as a single synchronized fabric. A total of four Fabric modules are required to build a complete Fabric in the OS-8800 system.
Separate paychecks for multicast • Generates ingress coupons, receives egress coupons for backpressure capability • Ingress coupon generation period: ~2 microseconds • Paycheck generation period: 32.125 microseconds • The BBUS provides access to individual Roma chips based on a chip index number •...
Roma Flow Control Coupon and paycheck thresholds are set for unicast and IPC queues (multicast is limited by Coronado for fabric ingress). These thresholds are used to generate ingress backpressure and paycheck messages. Each Roma ASIC participating as an operational bit slice is responsible for 2 NIs according to the scheme: bit slice n is responsible for NI slots 2(n+1) and 2(n+1)-1.
Fabric Slot Extraction This state assumes that there are five fabric slots occupied, with one fabric card serving as the redundant card. Roma Driver will have already set up the operational Roma chip set to use built-in hot swap handling. Upon extraction of fabric card, most of the hot swap handling is done by Roma ASIC logic. The performs the following tasks when a fabric card is removed: Recover the new fabric slot set state since the operational fabric may change.
Chassis Management Module (CMM) Framing Error The Framing Error event represents an event where the Roma ASIC detects an error in a packet header on ingress. Since this will potentially corrupt the buffer system, a free list rebuild is required when this is detected.
OS7000 CMM The CMM for OS7000 contains the following: System Processor • Ultra Space 11e (400MHz) • 64MB SDRAM Memory • Switching Fabrics • 4 Switching fabrics, Nantucket, in OS-7700 • 8 Switching fabrics, Nantucket, in OS-7800 • Management • DB-9 console/modem Port •...
Chassis Management Module (CMM) Functional Description of CMM Software and Configuration management is implemented in such a way to provide the operator with: Flexibility • Resiliency • And to minimize the service interruption during the update of a network. • 2 Software versions are stored into flash: •...
Boot ROM Sparc Processor executes the Sparc Boot ROM code from flash Memory in the protected memory. • Performs minimum diagnostic tests of the Sparc Processor • Verifies memory used by Sparc Processor is fine • If the diagnostic tests find an error, the CMM/Fabric LED will flash to indicate the error and the •...
Chassis Management Module (CMM) MiniBoot MiniBoot contains VX Works Operating system Performs the following tasks: Post Mortem Dump (PMD) Processing to save as much diagnostic as possible in the FFS after a system • crash. Hardware diagnostics to determine if all the boards are operational at boot time. •...
Page 355
Flash Files /flash/boot Boot ROM (Hidden & Protected) MiniBoot 1-Default 2-Backup (Write Protected) Boot.cfg ……… Contains boot /flash /working /certified /log /pmd OmniSwitch Troubleshooting Guide Execute Read/Write Read Read/Write September 2005 Chassis Management Module (CMM) Running Version Miniboot Load Minimum VX Works with File system with PMD process.
Chassis Management Module (CMM) • Falcon Overall Block Diagram Nantuck Primary CMM Board Overall System Architecture Fabric resides on CMM • Number of Nantuckets can be 4 or 8 depending on the chassis • Fabric Bus is connected to the CMM as well as to the All NI •...
Page 357
• Eagle Overall Block Diagram Fabric resides independent of the CMM. It resides at the Back of the Chassis. CMM can fail over inde- • pendent of Switching Fabric and vice versa. Minimum of 4 Switching Fabric Modules (SFM) with 8 ROMS chips are required to operate. The •...
Packet Walk Packet Walk Packet Walk Principles All switching and routing is performed on the Ingress side of the switch. There are no address lookups • made on the Egress side. The Coronado combines the L2 switching and L3 routing logic into the same ASIC. •...
Specific Packet Flows Unknown L2 Source, Known L2 Destination The Catalina ASIC Packet arrives at Catalina. CRC check done. If valid CRC it is put on XYBUS to Coronado The Coronado ASIC The FIFO logic maintains queues of frames from both Xybus interfaces and selects an incoming frame •...
Unknown Destination Unknown Destination Known L2 Source, Unknown L2 Destination The Catalina ASIC Packet arrives at Catalina. CRC check done. If valid CRC it is put on XYBUS to Coronado The Coronado ASIC The FIFO logic maintains queues of frames from both Xybus interfaces and selects an incoming frame •...
The Coronado ASIC The Coronado receives the packet via the FBUS. • The Coronado then strips the 802.1q header that was added on ingress, if needed. • The Catalina ASIC Packet is then put on to the Xybus to be received by the Catalina. •...
Unknown Destination Unknown L3 DA The Coronado ASIC The FIFO logic maintains queues of frames from both Xybus interfaces and selects an incoming frame • for HRE processing. This FIFO is done on the on chip memory of the HRE. The parser logic selects fields from the frame to identify the protocol and find key values that are used •...
Page 363
Pseudo-CAM (PCAM) Provides Group ID (GID) Is DA Special ? Is DA Known? Egress Coronado forwards the packet to destination port, strips Q-tag, if needed. Known/Unknown SA and Known DA DA QID is unknown, QID is set to flood, PDI is set to unknown DA and ReQID is set for s/w for unknown DA S/W De Queue s packet.
Page 364
Unknown Destination DA is router MAC DA PCAM lookup, known ? Provides a Next_Hop_Header_Cache index NHHC index provides QID, ReQID & PDI DA is IGMP QID set ReQID to either Sent to multicast S/W for within processing the vlan or drop Egress Coronado sends to all the ports in the same vlan, strips tag if necessary page -40...
Hardware Buses on OmniSwitch 7700/7800/ 8800 Switches Xybus The interface between the MAC to Coronado, which is 0-1 Fbus The interface between the Coronado to the Fabric ASIC (Nantucket or Roma). For the OmniSwitch 7700 (Falcon half chassis) it is 0-7. For the OmniSwitch 7800 (Falcon full chassis) it is 0-15. For the OmniSwitch 8800 (Eagle) it is 0-63.
Bus Mapping on OmniSwitch 7700/7800/8800 Switches Bus Mapping on OmniSwitch 7700/7800/8800 Switches Xybus Mapping Each board type has own mapping, described for all existing board type below. OS7-ENI-C24 and OS8-ENI-C24: Single slice board, connects two Catalina MAC ASICs through • xybus0 and xybus1.
OS6624/6648 Architecture OmniSwitch 6XXX is a stackable version of the OS7/8XXX Switches. It provides pure Ethernet switch- ing for 10/100 and Gigabit ports, along with stackability. The key features of OmniSwitch 6XXX are: ELEMENT - One Standalone HAWK Unit (1 or 2 Intel Devices) •...
OS6624/6648 Architecture Hardware Architectural Overview OS6600 uses off the shelf Intel ASIC. Intel IXE2424 Switching ASIC is used to implement the required functionality. The key features of Intel IXE2424 are: Provides 24 10/100 FE ports and 4 GE ports. • L2 and L3 switching •...
Page 369
Address Memory 32 Bit PCI 66 MHz OmniSwitch Troubleshooting Guide 100MH IXE 2424 GM11/10 Bit Interface 125MHz External Interfaces to IXE 2424 September 2005 OS6624/6648 Architecture Packet Memory SMII 125 MHz MDIO Interface page -45...
OS6624/6648 Architecture Layer 2 Forwarding Packets enter the IXE2424 through the SMII & GMII/TBI pins from external PHY devices. First, the • MAC associated with the port the packet was received from processes the packet. The MAC checks the CRC to see if the packet is valid and also updates appropriate receive packet port statistics. In parallel, two things happen next.
Address Learning Address Learning in the IXE2424 is performed primarily in the software. The hardware provides a CAM interface to facilitate fast learning of addresses. On power-up, the switch does not know which addresses are associated with which ports. So, when the switch receives the first packet, the source address lookup fails.
OS6624/6648 Architecture Address Look-up Methodology Source and Destination Addresses are searched in parallel and then a sync process occurs to ensure both are completed. The IXE2424 uses a fast method of organizing and searching the address records to meet wire speed performance requirements. Record Entries (see Layer 2 Data Structures section below) contain the addresses.
3-Protocol Entry Used to allow different rules to be applied for different protocols associated with same address. Proto- • col Offset Register is used to identify which protocols will be used in the system. Up to 18 Layer 2 protocols are supported. There is a separate protocol entry for each protocol defined in the Protocol Offset Register Priority level of the packet for queuing •...
OS6624/6648 Architecture Layer 3 Forwarding Packets Arrived Header 64 Bytes Apply protocol based VLAN on Search the IP Source and Destination addresses in the Send the packet to CPU for address Learning Broadcast the packet to all the ports in the VLAN Apply Filter, Mirror, Priority, QOS and WRED Rules Find outgoing Port’s 802.1Q...
VLANs The IXE2424 supports VLANs based on: Ports. This is accomplished using Port Net ID Entry described below. • 802.1Q Tags. This tag is included in the packet. • Protocols. This is accomplished using the Protocol to VID Lookup Table described below. •...
OS6624/6648 Architecture Tag Net ID Entry Contains related information like tagged set and member set. A 4x32 bit entry is present for every • possible VLAN (4096 supported). VLAN Tag Valid. Indicates if this VLAN tag has been configured by the CPU. If not, all packets with •...
Rules-Based Priority Several types of priority rules are supported. Each rule is associated with a specific device priority. If a • packet satisfies a certain priority rule, it is assigned the device priority corresponding to that rule. Global Priority Rules enable assignment of specific device priority to all packets from a specific source •...
CMM Functionality for OS6600 CMM Functionality for OS6600 The overall software architecture of Falcon is retained for Hawk. The user will perceive the system as a virtual chassis; where one element is elected as the primary CMM, another as the secondary CMM and the rest of the elements as NI.
Page 379
The chassis like, or V-Chassis, strategy allows the system to keep the same management interface. Since on Falcon/Eagle, the management uses the notion of slot, this notion of slot is retained for Hawk. Since there is no chassis, we need to provide a means by which a slot number is assigned to an element. The current strategy is to have the user assigning a slot number, via a push button on the front panel, located below the LCD display.
Page 380
CMM Functionality for OS6600 In idle mode the role of Chassis Supervision is limited to monitoring the temperature, power supply, • fan(s), handling of flash synchronization and takeover. The following services are provided when running in idle mode: Prober needs to monitor the temperature and report a temperature rising over a defined warning thresh- •...
Page 381
The process of multiple CMMs is composed of following steps: Bootup • Role election • Lowest slot number is elected as Primary • Following number is elected as secondary • Redundancy • Failure of primary • Secondary takeover • Lowest idle number become secondary •...
OS6600 IPC Communication OS6600 IPC Communication IPC software interface is unchanged. IPC provides the following services: NI-NI communication • CMM-NI communication • CMM is the primary or active • CMM-CMM communication • Each element in a stack is addressable from its CMM perspective. Different Types of Sockets are: •...
OS6600 BOOT Sequence Different from Falcon/Eagle • No NI code to download • Each element is independent from the other • For instance it is possible to have a primary which has not the lowest slot number • The following figures illustrate the bootup process: Load Stack Base...
Page 384
OS6600 BOOT Sequence Get the Role of the Stack-Primary, Secondary or Idle • Start NI supervision and Prober • Start Stack Manager • Get the stack port Configuration from the ESM Driver • IXE 2424 Supervisio ’ Dispatche Manager NI Supervision performs the following Tasks: •...
Page 385
OS6600 BOOT Sequence Chassis Supervisio Daughter Tasks applicatio NI powered on applicatio Driver Start message Supervisio applicatio ’ applicatio applicatio applicatio NI supervision and Prober Task. NI Supervision task in turn starts the NI applications. • OmniSwitch Troubleshooting Guide September 2005 page -61...
Page 387
B Debug Commands This chapter documents the following Command Line Interface (CLI) debug commands. cates that the text is hypertext-linked to additional documentation for that command. 802.1Q Debug Commands DVMRP Debug Commands IP Debug Commands DHCP Relay Debug Commands Multicast Routing Debug Commands OSPF Debug Commands PIM-SM Debug Commands...
output definitions Aggregate/Slot Status Port Status General Info Hardware Info Release History Release 5.1; command was introduced. Related Commands MIB Objects OmniSwitch Troubleshooting Guide Whether the slot or aggregate link is actively running. Whether the port is actively running. Provides general information on the modules in the chassis, including module type, number of ports, and ASIC.
Page 394
ip dvmrp debug-level Defines the level of debugging for DVMRP protocol on the switch. ip dvmrp debug-level level Syntax Definitions level Defaults parameter level Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines When the debug level is set to 0, DVMRP debug logging is turned off. Examples ->...
Page 395
ip dvmrp debug-type Enables or disables DVMRP debugging for a specified message type, or for all message types. Note. Debugging for a specified message type will only be enabled if its debug level is a value greater than zero (i.e., 1–255). For information on specifying the debug level, refer to the ip dvmrp debug-level command.
Page 396
Defaults parameter message_type Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines Use the no form of the command to disable debugging for the specified item. • Reminder: Debugging for a specified message type will only be enabled if its debug level is a value •...
show ip dvmrp debug Displays the current level of debugging for DVMRP protocol on the switch, as well as the current DVMRP debugging status for all messages types. show ip dvmrp debug Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines The administrative debugging status for message types displayed in the table are determined by the •...
Page 398
output definitions (continued) Grafts IGMP IPMRM Init Misc Probes Prunes Routes Time Release History Release 5.1; command was introduced. Related Commands ip dvmrp debug-level ip dvmrp debug-type page -12 The current debugging status for DVMRP Graft processing. Options include on or off. The current debugging status for DVMRP Internet Group Management Protocol (IGMP) packet processing.
debug ip packet Enables/disables/configures the IP packet debug feature. This command is generally used only when working with a field engineer to debug a problem on the switch. debug ip packet [start] [timeout seconds] [stop] [direction {in | out | all}] [format {header | text | all}] [output {console | file filename}] [board {cmm | ni [1-16] | all | none} [ether-type {arp | ip | hex [hex_number] | all}] [ip-address ip_address] [ip-address ip_address] [ip-pair [ip1] [ip2]] [protocol {tcp | udp | icmp | igmp | num [integer] | all}] [show-broadcast {on | off}] show-multicast {on | off}]...
Page 400
| udp | icmp | igmp | num | all on | off on | off Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines If you use the basic command to start debug (debug ip packet start) the switch will use default param- •...
Page 401
Related Commands debug ip level OmniSwitch Troubleshooting Guide Configures IP debug level. This command allows you to set the level (amount) of information displayed. September 2005 page -15...
Page 402
Syntax Definitions level Defaults parameter level Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines The debug level applies to the debug configuration set with the set different levels for different configurations. Examples -> debug ip level 1 Release History Release 5.1;...
Page 403
Returns IP packet debug options to default values. debug ip packet default Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines “Defaults” on page B-14 Examples -> debug ip packet default Release History Release 5.1;...
Page 404
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command is used to display IP debug configuration parameters. To start IP debugging you must enter the start keyword. Examples ->...
Page 405
Release History Release 5.1; command was introduced. Related Commands debug ip packet Configures IP packet debug. OmniSwitch Troubleshooting Guide September 2005 page -19...
Page 406
debug ip mroute debug-level Configures the Mutlicast Routing debug level. debug ip mroute debug-level level Syntax Definitions level Defaults parameter level Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines Higher debug-levels will include all messages that correspond to a lower value. For example, a debug level of 2 will display all messages for level 1 and level 2.
Page 408
Defaults parameter level Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command allows you to select the granularity at which you wish to view debugging information. Currently, in OSPF, there are three levels available: 10–Only critical errors and warnings.
Page 409
ip ospf debug-type Configures the type of OSPF functionality to debug. ip ospf debug-type [error] [warning] [state] [recv] [send] [flood] [spf] [lsdb] [rdb] [age] [vlink] [redist] [summary] [dbexch] [hello] [auth] [area] [intf] [mip] [info] [setup] [time] [tm] [all] no ip ospf debug-type [error] [warning] [state] [recv] [send] [flood] [spf] [lsdb] [rdb] [age] [vlink] [redist] [summary] [dbexch] [hello] [auth] [area] [intf] [mip] [info] [setup] [time] [tm] [all] Syntax Definitions error...
Page 410
| hello| auth | area | intf | mip | info | setup | time | tm | Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines The debug command allows you to enable debugging on various OSPF functions. These messages can •...
Page 411
Related Commands ip ospf debug-level show ip ospf debug MIB Objects ALAOSPFDEBUGCONFIG alaOspfDebugError alaOspfDebugWarning alaOspfDebugState alaOspfDebugRecv alaOspfDebugSend alaOspfDebugFlood alaOspfDebugSPF alaOspfDebugLsdb alaOspfDebugRdb alaOspfDebugAge alaOspfDebugVlink alaOspfDebugRedist alaOspfDebugSummary alaOspfDebugDbexch alaOspfDebugHello alaOspfDebugAuth alaOspfDebugArea alaOspfDebugIntf alaOspfDebugMip alaOspfDebugInfo alaOspfDebugSetup alaOspfDebugTime alaOspfDebugTm alaOspfDebugAll OmniSwitch Troubleshooting Guide Configures OSPF debugging level. Displays current OSPF debug level and types.
Page 412
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command is used to display the debugging information currently enabled for the OSPF router. • See the related commands sections below to modify the list.
Page 413
output definitions Debug Level error warning state recv send flood lsdb vlink redist summary dbexch hello auth area intf info setup time Release History Release 5.1; command was introduced. Related Commands ip ospf debug-level ip ospf debug-type OmniSwitch Troubleshooting Guide The granularity of the debug messages.
Page 415
ip pimsm debug-level Defines the level of PIM-SM debug messages that are generated. ip pimsm debug-level level Syntax Definitions level Defaults parameter level Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines When the debug level is set to 0, PIM-SM debug logging is turned off. Examples ->...
Page 416
ip pimsm debug-type Configures the type(s) of PIM-SM debug messages to display. ip pimsm debug-type message_list no ip pimsm debug-type message_list Syntax Definitions message_list supported message types assert bootstrap error hello igmp ipmrm init joinprune misc route time Defaults page -30 Specifies the type(s) of PIM-SM messages to be debugged.
Page 417
Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines The message-types specified in the command line will only be displayed if the debug level has been set • to a number greater than zero (i.e., 1–255). For information on specifying the debug level, refer to the ip pimsm debug-level command on page The syntax all can be used to easily turn on/off all message types (e.g., ip pimsm debug-type all or •...
Page 418
show ip pimsm debug Displays the current PIM-SM debug levels and types. show ip pimsm debug Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines The debug types displayed in the table are determined by the page B-30. To configure debug levels, refer to the Examples ->...
Page 419
output definitions (continued) error hello igmp init ipmrm joinprune misc route time Release History Release 5.1; command was introduced. Related Commands ip pimsm debug-level ip pimsm debug-type OmniSwitch Troubleshooting Guide The current state of messages related to Candidate Rendezvous Point (C-RP).
Page 420
Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Use the no form of the command to delete a debug type. • RIP must be enabled on the switch with the ip rip status CLI command before you can configure the •...
Page 421
Release History Release 5.1; command was introduced. Related Commands ip rip debug-level show ip rip debug MIB Objects alaRipLogTable alaRipDebugType OmniSwitch Troubleshooting Guide Configures RIP debugging level. Displays the current RIP debug levels and types. September 2005 page -35...
Page 422
Defaults parameter level Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines RIP must be enabled on the switch with the ip rip status CLI command before you can configure the • debug level. The debug level applies to all debug types that are configured. You cannot set different levels for each •...
Page 423
Displays the current RIP debug levels and types. show ip rip debug Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> show ip rip debug Debug Level Types/Sections error = on...
Page 424
Related Commands ip rip debug-level ip rip debug-type MIB Objects dispDrcRipDebug page -38 Configures RIP debugging level. Configures the type of RIP messages to debug. OmniSwitch Troubleshooting Guide September 2005...
Page 425
MDIX, IPC pools, and other phy-related information. debug slot information slot Syntax Definitions slot Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines The debug slot information command combines the functions of the show interfaces counters, debug interfaces interfaces...
Page 426
InPauseFrames = ######################################### debug interfaces 1 Led ######################################### Slot/Port Activity -----------+----------+-------- normal normal normal normal normal normal normal normal normal 1/10 normal 1/11 normal 1/12 normal 1/13 normal 1/14 normal 1/15 normal 1/16 normal 1/17 normal 1/18 normal 1/19 normal 1/20 normal 1/21...
Page 427
######################################### debug ipc pools slot 1 ######################################### IPC Pools slot 1, slice 0: UrgentPool: Full size is 256, remaining: 256 In socket queues: 0 Not queued: 0: In DMA queues: 0 ControlPool: Full size is 1024, remaining: 1023 In socket queues: 0 Not queued: 1: In DMA queues: 0 NormalPool: Full size is 256, remaining: 255 In socket queues: 0 Not queued: 1:...
18 : slot/port( 1/22): Phy ID : 3000 7809 10 : 8000 18 : slot/port( 1/23): Phy ID : 3000 7809 10 : 8000 18 : slot/port( 1/24): Phy ID : 3000 7809 10 : 8000 18 : ######################################### debug interfaces 1 mac ######################################### ERROR: Type <0>...
Page 431
runt runt sz = flowstate = 0 flowmode slot/port = ( 1/3 ) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:2a prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:2a long long sz = runt runt sz = flowstate = 0 flowmode...
Page 432
long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/7 ) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:2e prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:2e long long sz = runt runt sz =...
Page 433
pause SA = 0x 0:d0:95:6a:5f:31 long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/11) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:32 prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:32 long long sz = runt...
Page 434
trap pause SA = 0x 0:d0:95:6a:5f:35 long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/15) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:36 prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:36 long long sz = runt...
Page 435
trap pause SA = 0x 0:d0:95:6a:5f:39 long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/19) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:3a prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:3a long long sz = runt...
Page 436
1553 trap pause SA = 0x 0:d0:95:6a:5f:3d long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/23) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:3e prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:3e long long sz =...
debug interfaces set backpressure Enables and disables fabric back pressure on a Network Interface (NI) or an entire chassis. debug interfaces set [slot] backpressure {enable | disable} Syntax Definitions slot enable disable Defaults parameter enable | disable Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines If the slot number is not specified then the switch back pressure feature will be enabled or disabled on an entire chassis.
debug interfaces backpressure Displays if fabric back pressure is enabled or disabled on a Network Interface (NI) or an entire chassis. debug interfaces [slot] backpressure Syntax Definitions slot Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines If the slot number is not specified then the switch back pressure state will be displayed for an entire chas- sis.
Related Commands debug interfaces set backpressure MIB Objects OmniSwitch Troubleshooting Guide Enables and disables fabric back pressure on an NI or an entire chassis. September 2005 page -53...
Page 440
Displays LED information. debug interfaces slot led Syntax Definitions slot Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug interfaces 1 led Slot/Port Activity -----------+----------+-------- normal normal normal normal normal normal normal...
Page 441
Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide September 2005 page -55...
Page 447
Phy ID : 3000 7809 10 : 8000 18 : slot/port( 1/23): Phy ID : 3000 7809 10 : 8000 18 : slot/port( 1/24): Phy ID : 3000 7809 10 : 8000 18 : Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide 78f7...
Page 448
Syntax Definitions slot The slot number of the Network Interface (NI) module. Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug interfaces 1 mac ERROR: Type <0> for Debug_mac is unknown Release History Release 5.1;...
Page 450
slot/port = ( 1/3 ) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:2a prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:2a long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/4 ) slice mac_chip index...
Page 451
slot/port = ( 1/7 ) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:2e prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:2e long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/8 ) slice mac_chip index...
Page 452
flowstate = 0 flowmode slot/port = ( 1/11) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:32 prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:32 long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/12) slice...
Page 453
runt runt sz = flowstate = 0 flowmode slot/port = ( 1/15) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:36 prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:36 long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/16)
Page 454
long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/19) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:3a prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:3a long long sz = runt runt sz = flowstate =...
Page 455
pause SA = 0x 0:d0:95:6a:5f:3d long long sz = runt runt sz = flowstate = 0 flowmode slot/port = ( 1/23) slice mac_chip index = 0x = 0x00:d0:95:6a:5f:3e prevLink 0 cur_Link admin auto d_bw d_duplex = 1553 trap pause SA = 0x 0:d0:95:6a:5f:3e long long sz = runt...
Page 456
{slot | slot/port} mac stats Syntax Definitions slot port Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug interfaces 1 mac stats ERROR: Type <3> for Debug_mac is unknown Release History Release 5.1; command was introduced.
Page 457
Displays MAC level register information. debug interfaces {slot | slot/port} mac port Syntax Definitions slot port Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug interfaces 1/1 mac port slot/port(01/01): 100 : 120 : Output fields are described below:...
Page 458
Displays MAC level information for nonport from the register values. debug interfaces {slot | slot/port} mac nonport Syntax Definitions slot port Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug interfaces 1/1 mac nonport slot/port(01/01): Mac Asic Version : 20 :...
Page 461
Related Commands debug ipc pools cmm debug ipc bbus debug ipc active sockets debug ipc active sockets slot OmniSwitch Troubleshooting Guide Displays IPC Pools on Chassis Management Modules (CMMs). Displays IPC pools for the Burst Bus with transmit and received count, Direct Memory Access errors, and parity errors.
Page 463
Related Commands debug ipc pools slot debug ipc bbus debug ipc active sockets debug ipc active sockets slot OmniSwitch Troubleshooting Guide Displays IPC pools on a particular slot. Displays IPC pools for the Burst Bus with transmit and received count, Direct Memory Access errors, and parity errors. Displays all the active sockets on Chassis Management Modules (CMMs).
Page 464
Displays IPC pools for the Burst Bus with transmit and received count, Direct Memory Access errors, and parity errors. debug ipc bbus Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug ipc bbus Slot|Enabled | Tx Cnt ----|--------|---------|---------|---------|---------|...
Page 465
5944| 5946| 5955| Release History Release 5.1; command was introduced. Related Commands debug ipc pools cmm debug ipc pools slot debug ipc active sockets debug ipc active sockets slot OmniSwitch Troubleshooting Guide 106369| 108400| 107128| Displays IPC Pools on Chassis Management Modules (CMMs). Displays IPC pools on a particular slot.
Page 466
Displays all the active sockets on Chassis Management Modules (CMMs). debug ipc active sockets Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug ipc active sockets Enter Process CB 328 Falsock control blocks found...
Page 467
Release History Release 5.1; command was introduced. Related Commands debug ipc pools cmm debug ipc bbus debug ipc pools slot debug ipc active sockets slot OmniSwitch Troubleshooting Guide Displays IPC Pools on Chassis Management Modules (CMMs). Displays IPC pools for the Burst Bus with transmit and received count, Direct Memory Access errors, and parity errors.
Page 468
Displays all the active sockets on a particular slot. debug ipc active sockets slot slot Syntax Definitions slot Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug ipc active sockets slot 11 93 Falsock control blocks found slot 11 CB ADR...
Page 470
debug fabric threshold Displays the threshold number for each fabric ASIC. In addition, it also displays the Unicast pay gener- ated internally using Pay algorithm and Coupons generated. debug fabric threshold Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches.
Page 475
Release History Release 5.1; command was introduced. Related Commands debug fabric status debug fabric stats OmniSwitch Troubleshooting Guide Displays the overall status of all the fabric ASICs. Displays the fabric ASIC statistics. September 2005 page -89...
Page 476
debug fabric status Displays the overall status of all the fabric ASICs. It displays the chip version and netlist version being used. In addition, it also displays if any internal or external interrupts were received. debug fabric status Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800...
Page 477
debug fabric stats Displays the fabric ASIC statistics. debug fabric stats Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches. • “Unicast In” should be equal to the “Unicast Out”. •...
Page 478
Release History Release 5.1; command was introduced. Related Commands debug fabric status Displays the overall status of all the fabric ASICs. page -92 OmniSwitch Troubleshooting Guide September 2005...
Page 479
debug fabric output Displays the fabric ASIC port number and the frame count. debug fabric output Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches. • Each fabric ASIC has 16 ports in case of 7800 and 8 port in case of 7700. •...
Page 482
Related Commands debug fabric status debug fabric stats page -96 Displays the overall status of all the fabric ASICs. Displays the fabric ASIC statistics. OmniSwitch Troubleshooting Guide September 2005...
Page 483
debug fabric mcvectors Displays the Fabric ASIC port number and the frame count for multicast packets. debug fabric mcvectors Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches. •...
Page 489
4004 1006 586d 4004 1004 Release History Release 5.1; command was introduced. Related Commands debug fabric status debug fabric stats OmniSwitch Troubleshooting Guide Displays the overall status of all the fabric ASICs. Displays the fabric ASIC statistics. September 2005 page -103...
Page 490
debug fabric input Displays the fabric ASIC port number, frame count, and error count. debug fabric input Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches. • Each fabric ASIC has 16 ports in case of 7800 and 8 port in case of 7700. •...
Page 493
Related Commands debug fabric status debug fabric stats OmniSwitch Troubleshooting Guide Displays the overall status of all the fabric ASICs. Displays the fabric ASIC statistics. September 2005 page -107...
Page 494
debug fabric fbus Displays the fabric ASIC port number and the synchronization status for all the FBUSs. debug fabric fbus Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches. •...
Page 497
Related Commands debug fabric status debug fabric stats OmniSwitch Troubleshooting Guide Displays the overall status of all the fabric ASICs. Displays the fabric ASIC statistics. September 2005 page -111...
Page 498
debug fabric errors Displays the errors detected for all the fabric ASICs on a switch. debug fabric errors Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is only not for OS-6600 series of switches. Examples - >debug fabric errors Nantucket Framing Parity...
Page 499
debug fabric calendars Displays the fabric ASIC port number, calendar number, and the calendar length. debug fabric calendars Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command is not valid for OS-6600 series of switches. • Each fabric ASIC has 16 ports in case of 7800 and 8 port in case of 7700.
Page 505
9-16 17-24 25-32 1- 8 9-16 17-24 25-32 1- 8 9-16 17-24 25-32 1- 8 9-16 17-24 25-32 1- 8 9-16 17-24 25-32 1- 8 9-16 17-24 Release History Release 5.1; command was introduced. Related Commands debug fabric status debug fabric stats OmniSwitch Troubleshooting Guide Displays the overall status of all the fabric ASICs.
Page 506
debug slb help Prints a list of all debug Server Load Balancing (SLB) options. debug slb help Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines Server Load Balancing (SLB) is not supported on OS-6600 series of switches. •...
Page 507
CMMTrace NITrace NIDebug Flags Traps SimServers ServerArp PacketLoss Kill Snapshot Certify Takeover Help Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide <level> <level> <slot> <slice> <enable=0|1> <enable=0|1> <clusterid> <serverid> <macaddr> <port> <percentage> <slot> <slice> <status=0|1> <resetni=0|1> September 2005 page -121...
Displays the HTTP session database. debug http sessiondb Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug http sessiondb Sess SessName Name TimeOut Status URL Name--&--StatMsg ---+----+-------------+---------+-------+---------------+-------------------------- ------- Current Active WebView Session: 0 Release History Release 5.1;...
Page 509
debug hre warn Enables and disables the Hardware Routing Engine (HRE) warning messages on a specific HRE. debug hre warn {enable | disable} slot/slice Syntax Definitions enable disable slot slice Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 510
debug hre trace Enables and disables Hardware Routing Engine (HRE) trace messages on a specific HRE. debug hre trace {enable | disable} slot/slice Syntax Definitions enable disable slot slice Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 511
debug hre pcam Displays all the PCAM entries known on a particular slot and slice. debug hre pcam slot/slice There are two possible arguments to this command - <start> and <count>. <start> is the PCAM index to start with; default is 0. <count> is the number of entries to display; max is 24, default is 12. Syntax Definitions slot slice...
Page 513
debug hre pcam verbose Displays all the PCAM entries known on a particular slot and slice. debug hre pcam verbose slot/slice [start count] Syntax Definitions slot slice start count Defaults parameter start count Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches.
Page 515
debug hre pcam mode range This command displays the range of PCAM indices assigned to each mode. debug hre pcam mode range slot/slice Syntax Definitions slot slice Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 516
debug hre history Displays the history of the Hardware Routing Engine (HRE). debug hre history slot/slice Syntax Definitions slot slice Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 517
Related Commands OmniSwitch Troubleshooting Guide September 2005 page -131...
Page 518
debug hre error Enables/Disables error messages from Hardware Routing Engine (HRE) support software on a Chassis Management Module (CMM) for a specific slot/slice. debug hre error {enable | disable} slot/slice Syntax Definitions enable disable slot slice Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches.
Page 519
debug hre debug Enables/Disables debug messages from Hardware Routing Engine (HRE) support software on a Chassis Management Module (CMM) for a specific slot/slice. debug hre debug {enable | disable} slot/slice Syntax Definitions enable disable slot slice Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches.
Page 520
debug hre cmm warn Enables/Disables warning messages from Hardware Routing Engine (HRE) support software on a Chassis Management Module (CMM). debug hre cmm warn {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 521
debug hre cmm trace Enables/Disables trace messages from Hardware Routing Engine (HRE) support software on a Chassis Management Module (CMM). debug hre cmm trace {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 522
debug hre cmm error Enables/Disables error messages from Hardware Routing Engine (HRE) support software on a Chassis Management Module (CMM). debug hre cmm error {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 523
debug hre cmm debug Enables/Disables debug messages from Hardware Routing Engine (HRE) support software on a Chassis Management Module (CMM). debug hre cmm debug {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-66000 series does not have a HRE so this commands is not supported on these switches. Examples ->...
Page 524
Enables and disables health debugging. debug health {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug health enable Release History Release 5.1; command was introduced. Related Commands...
Page 525
CPU’s health if parameters are specified. debug health cpu [slot] Syntax Definitions slot Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples If no parameters are specified: -> debug health cpu Device Level Cpu Utilization...
Page 526
Release History Release 5.1; command was introduced. Related Commands debug health Enables and disables health debugging. page -140 OmniSwitch Troubleshooting Guide September 2005...
Page 527
Displays health of receive utilization on a particular slot or slot/port. debug health rx [slot[/port]] Syntax Definitions slot port Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples A slot number is specified: -> debug health rx 2 Slot 2 Input Utilization SECONDS:...
Page 528
Related Commands page -142 OmniSwitch Troubleshooting Guide September 2005...
Page 529
[slot] Syntax Definitions slot Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples No slot number is specified: -> debug health memory Device Level Memory Utilization...
Page 530
Release History Release 5.1; command was introduced. Related Commands page -144 OmniSwitch Troubleshooting Guide September 2005...
Page 531
Displays the GMAP flags and information about GMAP entries in GMAP database. debug gmap flags Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug gmap flags GMAP Holdtime Interval (minutes)
Page 532
Enables and disables the flow control for the console. debug console flow control {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug console flow control enable Release History Release 5.1; command was introduced.
Page 533
Displays the current flow control status. debug console show flow control Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug console show flow control Flow Control: Enabled Output fields are described below:...
Page 534
Enables and disables the command-info mode. debug command-info {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug command-info enable CLI command info mode on Release History Release 5.1; command was introduced.
Page 535
Displays the current information about the session. debug clishell data Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug clishell data Session Id EUPM User CLI oper mode Def sub-parser...
Page 536
Verifies if the AMAP database is fine or not. debug amap database Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug amap database AMAP Debug database ok Output fields are described below:...
Page 537
Displays all the debug configurations. debug chassis show Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug chassis show Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide...
Page 538
debug chassis secondary emp Enables and disables the Ethernet Management Port (EMP) port on a secondary Chassis Management Module (CMM) debug chassis secondary emp {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines OS-6600 series does not have an EMP port. Examples ->...
Page 539
Enables and disables hello messages from the secondary Chassis Management Module (CMM) to the primary CMM. debug chassis hello {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug chassis hello disable Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide Enables hello messages.
Page 540
If the secondary CMM does not respond back in the interval, it is rebooted. debug chassis hello timers {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug chassis hello timers disable Release History Release 5.1; command was introduced.
Page 541
Enables and disables chassis auto-reboots after a fatal error. debug chassis auto-reboot {enable | disable | on | off} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug chassis auto-reboot enable Release History Release 5.1; command was introduced.
Page 542
Enables and disables Network Interface (NI) module auto-reboots after a fatal error. debug chassis auto-reboot ni {enable | disable} Syntax Definitions enable disable Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug chassis auto-reboot ni enable Release History Release 5.1; command was introduced.
Page 543
Displays the information about all the ports with their IfIndex and the VLAN membership. debug vlan vpas Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug vlan vpas port vlan...
Page 549
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines All the ports shown as “+” are in use. If a “+” appears under mobile column then it indicates that the port has been configured as mobile port.
Page 551
16/9 16/10 16/11 16/12 16/13 16/14 16/15 16/16 16/17 16/18 16/19 16/20 16/21 16/22 16/23 16/24 Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide September 2005 page -165...
Page 552
Displays the rules configured on the switch for Group Mobility. debug vlan rule database Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug vlan rule database IP NETWORK RULES ssz=2...
Page 553
Network Interface (NI) modules for synchronizing the rules configured on the CMM. debug vlan rule communication Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines GlobSlice indicates the slot number. • If an NI is not present in the chassis then the state will appear as dead.
Page 555
Network Interface (NI) modules. debug vlan communication Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines GlobSlice indicates the slot number. • If an NI is not present in the chassis then the state will appear as dead.
Page 558
Related Commands page -172 OmniSwitch Troubleshooting Guide September 2005...
Page 559
LED, and MAC related information debug port information slot/port Syntax Definitions slot port Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug port information 11/1 Release History Release 5.1; command was introduced. Related Commands OmniSwitch Troubleshooting Guide Specifies an interface slot number.
Page 560
Logs Layer 2 QoS events on the switch. Logs Layer 3 QoS events on the switch. Logs events for Network Address Translation policies. Not supported for the OmniSwitch 6624/6648. Logs events related to QoS ports. Logs QoS messages. Logs information whenever the switch classifies a flow; more details are provided if the log level is higher.
Page 561
Defaults By default basic information messages are logged (info). Error messages are always logged. Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Use this command to troubleshoot QoS events on the switch. • Use the no form of the command to change the type of messages that will be logged or to return debug- •...
Page 565
Displays sysTrace debug log information (e.g., sysTrace status, Application IDs with non-default Severity Level settings). debug systrace show Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug systrace show sysTrace is: - INITIALIZED...
Page 566
Related Commands debug ktrace debug ktrace appid level debug ktrace show debug ktrace show log debug systrace debug systrace watch debug systrace appid level debug systrace show log show log pmd MIB Objects page -180 Enables or disables kTrace logging. Adds or removes a kTrace capture level for a specified subsystem.
Page 567
debug systrace appid level Adds or removes a sysTrace capture level for a specified application ID (i.e., subsystem). debug systrace appid {app_id | integer} level {level | integer} debug systrace no appid app_id Syntax Definitions app_id appid integer Supported Application IDs and Numerical Equivalents 802.1q - 7 aaa - 20 amap - 18...
Page 568
Defaults parameter level Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines You may enter more than one application ID in the command line. Separate each application ID with a • space. Application IDs may be entered in any order.
Page 570
Output fields are described here: output definitions Timestamp AppID Trace Level Task Caller Session ID Comment Release History Release 5.1; command was introduced. Related Commands debug ktrace debug ktrace appid level debug ktrace show debug ktrace show log debug systrace debug systrace watch debug systrace appid level debug systrace show...
Page 571
Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines If no additional filter parameter is entered, all stored PMD file information will be displayed. Examples -> show log pmd filename PMD Version -> 102 File Dump Type ->...
Page 572
Record Type -> MemoryData 0 6e 31 3d 0 0 2c 4f 0 c7 58 0 58 40 0 d c3 20 0 4e 6f 24 0 9e c5 0 4 2c 9 3e d4 e dd 7e 3 18 42 50 3 43 8 d0 3 43...
Page 573
Registration Type ->Task Record Type -> UserDefined Address -> 1adcc38 Size -> 10 46 69 72 73 74 20 69 74 65 72 61 74 69 6f 6e ------------------------------------------------------------- Registration Type ->Task Record Type -> UserDefined Address -> 1adcc50 Size -> 11 53 65 63 6f 6e 64 20 69 74 65 72 61 74 69 6f 6e ------------------------------------------------------------- Registration Type ->Task...
Page 574
Output fields are described here: output definitions PMD Version File Dump Type Date Created Registration Type Application ID Record Type Address Size Task Number Task ID Task Priority Task Status Task Option Bits Original Entry Point of Task Size of Stack (bytes) Current Stack Usage (bytes) Maximum Stack Usage (bytes) The maximum amount of the stack used by the task registered with Task Error Status...
Page 576
The debug memory monitor show status command displays memory monitoring status information. debug memory monitor show status Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug pmd ni 3/0 Release History Release 5.1; command was introduced.
Page 587
Related Commands debug ktrace appid level debug ktrace show debug ktrace show log debug systrace debug systrace watch debug systrace appid level debug systrace show debug systrace show log show log pmd MIB Objects OmniSwitch Troubleshooting Guide Adds or removes a kTrace capture level for a specified subsystem. Displays current kTrace parameters.
Page 588
Displays current kTrace parameters (e.g., kTrace status, Application IDs with non-default Severity Level settings). debug ktrace show Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines Examples -> debug ktrace show kTrace is: - INITIALIZED - RUNNING - configured to TRACE CALLERS All applications have their trace level set to the level ‘info’...
Page 589
Related Commands debug ktrace debug ktrace appid level debug ktrace show log debug systrace debug systrace watch debug systrace appid level debug systrace show debug systrace show log show log pmd MIB Objects OmniSwitch Troubleshooting Guide Enables or disables kTrace logging. Adds or removes a kTrace capture level for a specified subsystem.
Page 590
debug ktrace appid level Adds or removes a kTrace capture level for a specified application ID (i.e., subsystem). debug ktrace appid {app_id | integer} level {level | integer} debug ktrace no appid app_id Syntax Definitions app_id appid integer Supported Application IDs and Numeric Equivalents 802.1q - 7 aaa - 20 bridge - 10...
Page 591
Defaults parameter level Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines You may enter more than one application ID in the command line. Separate each application ID with a • space. Application IDs may be entered in any order.
This chapter describes Technical Support Command Line Interface (CLI) show commands that create log files of the output from multiple standard CLI show commands. These log files can be transferred with FTP to a workstation for off-line analysis and troubleshooting. Note.
Page 596
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support.log in the /flash directory of the output produced by • the show hardware info, show chassis, show module long, show fan, show power, show...
Page 597
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_layer2.log in the /flash directory of the output • produced by the show interfaces, show interfaces accounting, show interfaces collisions, show vlan...
Page 598
See the OmniSwitch CLI Reference Guide for more information on show commands. • Server Load Balancing (SLB) is not supported on OmniSwitch 6624 and 6648 switches and therefore • the show ip slb command output is not relevant for these switches.
Page 599
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_rip.log in the /flash directory of the output produced • by the show ip rip, show ip rip routes, show ip rip redist-filter, show ip rip redist, show ip rip interface, show ip rip peer, and show ip rip debug CLI commands.
Page 600
show tech-support layer3 pimsm Creates a log file of the output of several Protocol-Independent Multicast Sparse Mode (PIM-SM) Command Line Interface (CLI) commands. show tech-support layer3 pimsm Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_pimsm.log in the /flash directory of the output •...
Page 601
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_ospf.log in the /flash directory of the output • produced by the show ip ospf, show ip ospf area, show ip ospf interface, show ip ospf neighbor,...
Page 602
show tech-support layer3 mroute Creates a log file of the output of several multicast routing Command Line Interface (CLI) commands. show tech-support layer3 pimsm Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_mroute.log in the /flash directory of the output •...
Page 603
Syntax Definitions Defaults Platforms Supported OmniSwitch 6624, 6648, 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_ipx.log in the /flash directory of the output produced • by the show ipx interface, show ipx default-route, show ipx route, show ipx servers, show ipx filter, show ipx type-20-propagation, show ipx packet-extension, and show ipx timers CLI commands.
Page 604
show tech-support layer3 dvmrp Creates a log file of the output of several Distance Vector Multicast Routing Protocol (DVMRP) Command Line Interface (CLI) commands. show tech-support layer3 dvmrp Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_dvmrp.log in the /flash directory of the output •...
Page 605
show tech-support layer3 bgp Creates a log file of the output of several Border Gateway Protocol (BGP) Command Line Interface (CLI) commands. show tech-support layer3 bgp Syntax Definitions Defaults Platforms Supported OmniSwitch 7700, 7800, 8800 Usage Guidelines This command creates a file called tech_support_bgp.log in the /flash directory of the output •...
Page 606
page -12 OmniSwitch Troubleshooting Guide September 2005...
D Modifying Files with The switch has a built in Unix text editor called VI. This section covers some basic VI commands and how to use VI to modify the IP address of the EMP (Ethernet Management Port), which is stored in the boot.params file. The boot.params file can also be modified via MiniBoot.
Page 608
Useful VI Commands Useful VI Commands The following are some useful VI commands: u - undo the last command. CTL/L -reprint current screen. CTL/F-pages forward one screen. CTL/B-pages back one screen. j -moves cursor down one line. k -moves cursor up one line. h - moves cursor back one character.
Dew - Deletes the current word. P - Puts back text from previous delete. yy-Puts the current line in buffer; leaves the current line intact. p-Places the line in the buffer after the current position of the cursor. ZZ-Exits VI and saves the changes. :q-quits VI session and does not save any of the changes.
Page 610
Sample VI Session page -4 OmniSwitch Troubleshooting Guide September 2005...
Page 612
show show ip rip redist command show spantree command 4-2, 7-4 show spantree ports command show stack topology command show swlog command show tech-support command 1-10, C-2 show tech-support layer2 command show tech-support layer3 bgp command show tech-support layer3 command show tech-support layer3 dvmrp command show tech-support layer3 ipx command show tech-support layer3 mroute command...