Alcatel-Lucent OmniSwitch 6860 Troubleshooting Manual
  1. Manuals
  2. Brands
  3. Alcatel-Lucent Manuals
  4. Network Router
  5. OmniSwitch 6860
  6. Troubleshooting manual

Alcatel-Lucent OmniSwitch 6860 Troubleshooting Manual

Hide thumbs Also See for OmniSwitch 6860:
Table of Contents

Advertisement

Quick Links

OmniSwitch OS6860/OS6900/OS10K Troubleshooting Guide
Part No.032996-00 Rev.A
AOS Release 7.X and 8.X
January 2015
OmniSwitch 6860/6900/10K

Troubleshooting Guide

Alcatel-Lucent Enterprise
26801 West Agoura Road, Calabasas, CA 91301
(818) 880-3500
www.alcatel-lucent.com
Copyright  1995-2015 Alcatel-Lucent
ALL RIGHTS RESERVED

Advertisement

Table of Contents

Troubleshooting

loading

Related Manuals for Alcatel-Lucent OmniSwitch 6860

Summary of Contents for Alcatel-Lucent OmniSwitch 6860

  • Page 1: Troubleshooting Guide

    OmniSwitch OS6860/OS6900/OS10K Troubleshooting Guide Part No.032996-00 Rev.A AOS Release 7.X and 8.X January 2015 OmniSwitch 6860/6900/10K Troubleshooting Guide Alcatel-Lucent Enterprise 26801 West Agoura Road, Calabasas, CA 91301 (818) 880-3500 www.alcatel-lucent.com Copyright  1995-2015 Alcatel-Lucent ALL RIGHTS RESERVED...

  • Page 2: Table Of Contents

    1.5. What is Not in this Manual? ........................5 1.6. How is the Information Organized? ......................5 1.7. Related Documentation..........................5 1.8. Before Calling Alcatel-Lucent’s Technical Assistance Center ..............6 2. Troubleshooting the Switch System ......................8 2.1. Introduction..............................8 2.2.
  • Page 3 23.1. Troubleshooting Port Mirroring ......................143 23.2. Troubleshooting port monitoring ......................144 24. Troubleshooting IPV6 ..........................146 24.1. IPv6 Routing ............................146 24.2. Troubleshooting DHCPv6 Relay ......................147 24.3. Troubleshooting a 6to4 Tunnel ......................148 Alcatel-Lucent Page 3 of 148...

  • Page 4: About This Guide

    Alcatel-Lucent Service and Support personnel. However, this guide is not intended for novice or first-time users of Alcatel-Lucent OmniSwitches. Misuse or failure to follow procedures in this guide correctly can cause lengthy network down time and/or permanent damage to hardware. Caution must be followed on distribution of this document.

  • Page 5: What Is Not In This Manual

    The following are the titles and descriptions of all the Release 8 and later OmniSwitch user guides: • OmniSwitch 6860/6860E Hardware Users Guide Complete technical specifications and procedures for OmniSwitch 6860, power supplies, fans, and Network Interface (NI) modules. •...

  • Page 6: Before Calling Alcatel-Lucent's Technical Assistance Center

    Before calling Alcatel-Lucent’s Technical Assistance Center (TAC), make sure that you have read through the appropriate section (or sections) and have completed the actions suggested for your system’s problem. Additionally, do the following and document the results so that the Alcatel-Lucent TAC can better assist you: •...
  • Page 7 SHASTA #-> Alcatel-Lucent Page 7 of 148...

  • Page 8: Troubleshooting The Switch System

    ------+-------------+------------+---------+------------------ CMM-B POWER ON e8:e7:32:9b:e2:6e SLOT-1 POWER ON 2.12 00:e0:b1:e4:c5:79 SLOT-2 POWER ON e8:e7:32:a5:d9:30 SLOT-5 POWER ON 00:e0:b1:e4:ae:a1 -> show powersupply Total Power Input Slot Power Used Voltage Type Status Location ---------+---------+--------+---------+--------+--------+----------- 1200 PWRSAVE Internal Alcatel-Lucent Page 8 of 148...

  • Page 9: Troubleshooting System On Os6860/E Switches

    CMM, the base chassis MAC address is displayed. For NI modules, the MAC address for the corresponding NI is displayed. Verify the power supply (or supplies). Check the power supply status, using the command: sno-lab-r1-6860> show powersupply Alcatel-Lucent Page 9 of 148...
  • Page 10 If none of the values are above the threshold, the next step is to attempt to isolate the problem to a particular NI, using command: 6860-> show health slot 1/1 Slot 1/ 1 Current 1 Min 1 Hr 1 Day Resources ----------------------+---------+-------+-------+------- Memory Receive Receive/Transmit Alcatel-Lucent Page 10 of 148...
  • Page 11 For setting up different log levels in switch log, please refer to the “Using Switch Logging” chapter in the appropriate OmniSwitch Network Configuration Guide. Alcatel-Lucent Page 11 of 148...

  • Page 12: Advanced Troubleshooting

    Open Flow CMM appMonCmm app-mon ofnid Open Flow NI appMonNiStub app-mon pmCmm Port mapping CMM Broadcom driver pmNi Port mapping NI Port Mirroring & Monitoring bcmd pmmCmm BFD CMM pmmnid Port Mirroring & Monitoring Ni Alcatel-Lucent Page 12 of 148...
  • Page 13 CPU and use knowledge of the network to determine whether or not the consumption is abnormal. Each process presents a task running in CPU. The meaning of process can be found in above figure. 4. Resolve the process causing high CPU Alcatel-Lucent Page 13 of 148...

  • Page 14: Memory Leak

    AOS Release 7.X and 8.X January 2015 Find the specific chapter in this troubleshooting guide for the task causing high CPU. It is possible that the task needs to be restarted. Before restarting the task, please contact Alcatel-Lucent customer support. 5. Identify abnormal traffic It's worthwhile to verify how many packets are trapped by the CPU due to FFP rules (packets may be trapped to the CPU due to other reasons—see chapter Packet Driver for more details).

  • Page 15: The Second Column Of Output Provides Changes Since The Previous Execution Of The Command.logs

    7645246 7645246 Ipv4: Rx Q Full Drops Ipv4: Rx 7640318 7640318 ……….. 2.7. The second column of output provides changes since the previous execution of the command.Logs Besides the swlog, the kernel log in /var/log/. Alcatel-Lucent Page 15 of 148...

  • Page 16: Troubleshooting Virtual Chassis

    Highest chassis priority -> virtual-chassis configured-chassis-priority 100 The highest number configured-chassis-priority will become the Master chassis. Without setting this value the lowest chassis identifier becomes the key value used to determine which switch will become the Master. Alcatel-Lucent Page 16 of 148...
  • Page 17 Invalid-Chassis-Id: The chassis is not operational in virtual chassis mode because no valid chassis identifier has been found in the configuration. Typically this means that the vcsetup.cfg file is corrupted, empty or contains an invalid (e.g. out of range) chassis identifier. Alcatel-Lucent Page 17 of 148...

  • Page 18: Basic Troubleshooting

    Displays a summary of the configured and operational parameters related to the virtual fabric links on the virtual chassis topology OS6860-> show virtual-chassis vf-link Primary Config Active Speed Chassis/VFLink ID Oper Port Port Port Vlan Type -------------------+----------+---------+-------+-------+---------+----------- 1/1/30 2/1/30 And per link Alcatel-Lucent Page 18 of 148...

  • Page 19: Advanced Troubleshooting

    VFL Intf Oper Status Down 01:03:57 NOK_09 VFL Intf Oper Status (@L5) Down 00:40:45 NOK_09 VFL LACP Status Down 01:03:57 NOK_14 VFL LACP Status (@L5) Down 00:40:45 NOK_14 VFL LACP Up -> Down 00:46:05 INFO_04 Alcatel-Lucent Page 19 of 148...
  • Page 20 This parameter provides the counter of how many times the virtual-fabric link operational status has transitioned from down to up since the switch was started. Depending on the specific operations you are performing on the system, this counter may Alcatel-Lucent Page 20 of 148...
  • Page 21 1/0 enable/disable all socket debug debugReac : 1/0 enable/disable reactor debug Public msg traces and statistics -------------------------------------------------------------------------------- > vcm_show_public_trace (start, count, detail) show range of public messages in the trace buffer. Use start = 0 and count = 0 Alcatel-Lucent Page 21 of 148...

  • Page 22: Scenarios

    Certify/Restore Status : CERTIFY NEEDED SYNCHRONIZATION STATUS Running Configuration : SYNCHRONIZED For example, you want to add virtual Chassis 7 into a stack of 6, you can issue the follow command to the new unit. Alcatel-Lucent Page 22 of 148...
  • Page 23  vcsetup.cfg is the virtual chassis configuration file, where VFL ports are defined. Note that vcboot.cfg is the same for both Master virtual chassis and Slave virtual chassis, but the vcsetup.cfg is Alcatel-Lucent Page 23 of 148...
  • Page 24 Down 1/1/30 2/1/29 Down 2/1/30 3/1/53 3/1/54 4/1/29 4/1/30 5/1/29 5/1/30 6/1/29 6/1/30 7/1/53 To check consistency of a virtual chassis: OS6860-10.255.13.68-> show virtual-chassis consistency Config Oper Oper Config Chas Chas Chas Hello Control Control Alcatel-Lucent Page 24 of 148...
  • Page 25 The above command will also show the internal ip address for every switch in the virtual-chassis. Remote Chassis Detection (RCD) occurs every 1 seconds between the switches in the virtual chassis when the EMP ports are configured. OS6860-10.255.13.68-> show virtual-chassis vf-link member-port Chassis/VFLink ID Chassis/Slot/Port Oper Is Primary Alcatel-Lucent Page 25 of 148...
  • Page 26 OmniSwitch OS6860/OS6900/OS10K Troubleshooting Guide Part No.032996-00 Rev.A AOS Release 7.X and 8.X January 2015 -------------------+------------------+----------+------------- 1/1/29 1/1/30 2/1/29 2/1/30 3/1/53 3/1/54 4/1/29 4/1/30 5/1/29 5/1/30 6/1/29 6/1/30 7/1/53 7/1/54 Alcatel-Lucent Page 26 of 148...

  • Page 27: Troubleshooting Switched Ethernet Connectivity

    VLAN. Also review the output of the command to verify there is nothing explicit in the configuration that would cause the problem, such as a deny ACL that could be found under the QoS subsection. Alcatel-Lucent Page 27 of 148...

  • Page 28: Verify Source Learning

    If everything checked appears to be valid, verify that this is not an ARP problem. On the end stations involved, enter a static MAC address for the device it is trying to communicate with. If connectivity is restored, please see Chapter 6 Troubleshooting ARP Alcatel-Lucent Page 28 of 148...

  • Page 29: Troubleshooting Source Learning/Layer 2 Forwarding

    Check the current MAC table size by using the below command to understand the number of MAC addresses learned on a switch. 6860-> show mac-learning summary Mac Address Table Summary: Domain Static Static-Multicast Bmac Dynamic Alcatel-Lucent Page 29 of 148...
  • Page 30 By default the MAC address aging time is set to 300 seconds. This can be viewed: 6860-> show mac-learning aging-time Mac Address Aging Time (seconds) = 300 This can be changed using the command: 6860-> mac-learning aging-time 500 6860-> show mac-learning aging-time Mac Address Aging Time (seconds) = 500 Alcatel-Lucent Page 30 of 148...

  • Page 31: Advanced Troubleshooting Scenarios

    Long Frame Size(Bytes) : 9216, Bytes Received 785209334, Unicast Frames : 255953, Broadcast Frames: 8787093, M-cast Frames 1420762, UnderSize Frames: 0, OverSize Frames: Lost Frames 0, Error Frames CRC Error Frames: 0, Alignments Err : Alcatel-Lucent Page 31 of 148...
  • Page 32 TCNs are received by the switch (at least 1 every 1 to 4 minutes) for one or more VLANs. In order to track down which VLAN and port the TCNs are received on (causing the global MAC aging timer to restart). Use the below command to find out which VLAN is receiving the TCNs Alcatel-Lucent Page 32 of 148...

  • Page 33: Mac Address Flapping

    -> show mac-learning mac-address 00:13:72:19:5e:1f Legend: Mac Address: * = address not valid, Mac Address: & = duplicate static address, Domain Vlan/SrvcId/ISId Mac Address Type Operation Interface -----------+---------------------+-------------------+---------------+------------+---------- VLAN 00:13:72:19:5e:1f dynamic bridging 2/1/15 -> show mac-learning mac-address 00:13:72:19:5e:1f Alcatel-Lucent Page 33 of 148...

  • Page 34: Bshell Troubleshooting

    5.3. bShell Troubleshooting Warning: Maintenance Shell commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel- Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
  • Page 35 Entering maintenance shell. Type 'exit' when you are done. TOR #-> bshell Entering character mode Escape character is '^]' BCM.0> search l2_entry mac_addr=0xe8e7322d2c47 Searching L2_ENTRY table indexes 0x0 through 0x1ffff... L2_ENTRY.ipipe0[5408]: <VPG_TYPE_1=1,VPG_TYPE=0,VPG_1=0x1ce,VPG=0x1d,VLAN_ID=1,VIF:L2MC_PTR=0x1d,VFI=1,VALID=1,T_1=1,TRI LL_NETWORK_RECEIVERS_PRESENT=0,TGID_1=0x1ce,TGID=0x1d,T=0,STATIC_BIT=1,SRC_DISCARD=0,SCP=0,RPE=0, RESERVED_1=0x322d2,REMOTE_TRUNK_1=1,REMOTE_TRUNK=0,PRI=0,PORT_NUM_1=0x4e,PORT_NUM=0x1d,PENDING=0, OVID=1,MODULE_ID_1=0xa3,MODULE_ID=0,MAC_BLOCK_INDEX=0,MAC_ADDR=0xe8e7322d2c47,LOCAL_SA=1,L2MC_PTR =0x1d,L2:VPG_TYPE=0,L2:VPG=0x1d,L2:VLAN_ID=1,L2:VFI=1,L2:TRILL_NETWORK_RECEIVERS_PRESENT=0,L2:TGI D=0x1d,L2:T=0,L2:STATIC_BIT=1,L2:SRC_DISCARD=0,L2:SCP=0,L2:RPE=0,L2:REMOTE_TRUNK=0,L2:PRI=0,L2:PO RT_NUM=0x1d,L2:PENDING=0,L2:MODULE_ID=0,L2:MAC_BLOCK_INDEX=0,L2:MAC_ADDR=0xe8e7322d2c47,L2:L2MC_P TR=0x1d,L2:KEY=0x7473991696238008,L2:DUMMY_INDEX=0,L2:DST_DISCARD=0,L2:DEST_TYPE=0,L2:DESTINATION =0x1d,L2:DATA=0x82000001d,L2:CPU=0,L2:CLASS_ID=0,L2:ASSOCIATED_DATA=0x82000001d,KEY_TYPE=0,KEY=0x 7473991696238008,IVID=0xc47,HITSA=1,HITDA=0,EVEN_PARITY=1,DUMMY_INDEX=0,DST_DISCARD=0,DEST_TYPE_1 Alcatel-Lucent Page 35 of 148...
  • Page 36 To recover the customer network from the issue state, change the static_bit value back to zero. BCM.0> mod l2_entry 5408 1 static_bit=0 BCM.0> search l2_entry mac_addr=0xe8e7322d2c47 Searching L2_ENTRY table indexes 0x0 through 0x1ffff... L2_ENTRY.ipipe0[5408]: <VPG_TYPE_1=1,VPG_TYPE=0,VPG_1=0x1ce,VPG=0x1d,VLAN_ID=1,VIF:L2MC_PTR=0x1d,VFI=1,VALID=1,T_1=1,TRI LL_NETWORK_RECEIVERS_PRESENT=0,TGID_1=0x1ce,TGID=0x1d,T=0,STATIC_BIT=0,SRC_DISCARD=0,SCP=0,RPE=0, RESERVED_1=0x322d2,REMOTE_TRUNK_1=1,REMOTE_TRUNK=0,PRI=0,PORT_NUM_1=0x4e,PORT_NUM=0x1d,PENDING=0, OVID=1,MODULE_ID_1=0xa3,MODULE_ID=0,MAC_BLOCK_INDEX=0,MAC_ADDR=0xe8e7322d2c47,LOCAL_SA=1,L2MC_PTR =0x1d,L2:VPG_TYPE=0,L2:VPG=0x1d,L2:VLAN_ID=1,L2:VFI=1,L2:TRILL_NETWORK_RECEIVERS_PRESENT=0,L2:TGI D=0x1d,L2:T=0,L2:STATIC_BIT=0,L2:SRC_DISCARD=0,L2:SCP=0,L2:RPE=0,L2:REMOTE_TRUNK=0,L2:PRI=0,L2:PO RT_NUM=0x1d,L2:PENDING=0,L2:MODULE_ID=0,L2:MAC_BLOCK_INDEX=0,L2:MAC_ADDR=0xe8e7322d2c47,L2:L2MC_P TR=0x1d,L2:KEY=0x7473991696238008,L2:DUMMY_INDEX=0,L2:DST_DISCARD=0,L2:DEST_TYPE=0,L2:DESTINATION =0x1d,L2:DATA=0x1d,L2:CPU=0,L2:CLASS_ID=0,L2:ASSOCIATED_DATA=0x1d,KEY_TYPE=0,KEY=0x74739916962380 08,IVID=0xc47,HITSA=1,HITDA=0,EVEN_PARITY=0,DUMMY_INDEX=0,DST_DISCARD=0,DEST_TYPE_1=3,DEST_TYPE=0 ,DESTINATION_1=0x51ce,DESTINATION=0x1d,DATA=0x1d,CPU=0,CLASS_ID=0,ASSOCIATED_DATA=0x1d> BCM.0> Alcatel-Lucent Page 36 of 148...

  • Page 37: Troubleshooting Arp

    To troubleshoot ARP the first step is to verify the MAC address of the SRC machine and DST machine are learned on the correct port in the correct VLAN. OS6860-> show mac-learning mac-address 00:13:72:19:5e:1f Legend: Mac Address: * = address not valid, Alcatel-Lucent Page 37 of 148...
  • Page 38 ARP request not reaching the switch, which may be because: - The workstation is not sending an ARP reply. - The workstation is not able to understand the ARP request. - The ARP response might have been corrupted. Alcatel-Lucent Page 38 of 148...

  • Page 39: Advanced Arp Troubleshooting

    AOS switch to generate ARP traffic at high rate due to continous requests for unresolved ARPs. 6.2. Advanced ARP Troubleshooting Warning: Maintenance Shell and debug commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel-Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
  • Page 40 : 42. arp send rep : 0. arp recv req : 0. arp recv rep : 0. IPNI-VRF-0> The below command to see number of gratuitous ARP statistics and other errored ARP packets. Alcatel-Lucent Page 40 of 148...
  • Page 41 Entering maintenance shell. Type 'exit' when you are done. SHASTA #-> cat /proc/alv4/stats gratarp = 1 gratarp err = 0 rcv = 0 rcv bad ifindex = 0 rcv missing ifindex = 0 dump drop = 0 dump size = 0 Alcatel-Lucent Page 41 of 148...

  • Page 42: Troubleshooting Spanning Tree

    This can be caused by dropped BPDUs (to be discussed later), a bridge that insists it is root regardless of received BPDUs, or a physical link going in and out of service. Alcatel-Lucent Page 42 of 148...
  • Page 43 Number of Status Change: 1, Type : Ethernet, SFP/XFP : N/A, : Disabled, Link-Quality : N/A, MAC address : e8:e7:32:b3:49:18, BandWidth (Megabits) 1000, Duplex : Full, Autonegotiation [ 1000-F 100-F 100-H 10-F 10-H ], Long Frame Size(Bytes) : 9216, Alcatel-Lucent Page 43 of 148...

  • Page 44: Advanced Troubleshooting

    VlanMgrCmm subapp all level debug2 swlog appid portMgrNi subapp all level debug2 swlog appid VlanMgrNi subapp all level debug2 Toggle the port state to recreate the issue. Reduce the logging level to ‘info’ and gather SWLOG outputs: Alcatel-Lucent Page 44 of 148...
  • Page 45 Stopped Troubleshooting in Maintenance Shell Commands Warning: Maintenance Shell commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel-Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
  • Page 46 TX: All the BPDU sent by the swith  Bpdu - 802.1d BPDUs  RBpd - 802.1w BPDUs  Flg80 - BPDUs with flag 80 set (Topology Change Acknowledgement)  Flg01 - BPDUs with flag 01 set (Topology Change) Alcatel-Lucent Page 46 of 148...
  • Page 47 Forward: ge0,hg BCM.0> OS6860-> show spantree ports blocking Vlan Port Oper Status Path Cost Role Note -----+-------+------------+---------+-------+--------- 1/1/3 OS6860-> show spantree ports forwarding Vlan Port Oper Status Path Cost Role Note -----+-------+------------+---------+-------+--------- 1/1/1 FORW ROOT Alcatel-Lucent Page 47 of 148...

  • Page 48: Troubleshooting Link Aggregation

     LACP BPDUs are tunneled by default on UNI ports. OS6860-> show ethernet-service uni-profile default-uni-profile Profile Name 802.1x 802.3ad 802.1ab MVRP AMAP -------------------+--------+--------+---------+---------+--------+--------- default-uni-profile tunnel tunnel tunnel tunnel tunnel tunnel Alcatel-Lucent Page 48 of 148...

  • Page 49: Basic Troubleshooting

    RDBGC0.ge28 g drop_pkt_cnt_ing.ge28 __________________________________________________________________ 8.1. Basic Troubleshooting show linkagg -> show linkagg Number Aggregate SNMP Id Size Admin State Oper State Att/Sel Ports -------+----------+--------+----+-------------+-------------+--------+----- Static 40000001 8 ENABLED Dynamic 40000002 4 ENABLED DOWN Alcatel-Lucent Page 49 of 148...

  • Page 50: Advanced Troubleshooting

    512 Gather logs: show configuration snapshot linkagg show linkagg port show log swlog | grep LACP show log swlog | grep linkagg The debug logs should be disabled after gathering outputs: Alcatel-Lucent Page 50 of 148...
  • Page 51 Troubleshooting in Maintenance Shell Warning: Maintenance Shell commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel-Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
  • Page 52 +7,210 R127.ge28 5,597 +5,597 R255.ge28 111,039 +111,039 Dump RDBGC0 counter 3 times in 30 second interval: BCM.0> g RDBGC0.ge28 RDBGC0.ge28[1][0x38002b1e]=0x1d3cf: <EVEN_PARITY=0,COUNT=0x1d3cf> Dump DROP_PKT_CNT_ING counter 3 times in 30 second interval: BCM.0> g drop_pkt_cnt_ing.ge28 DROP_PKT_CNT_ING.ge28[3][0x8001001d]=0: <COUNT=0> Alcatel-Lucent Page 52 of 148...

  • Page 53: Troubleshooting Bootp/Dhcp/Udp Relay

    IPv6 address by appending its 64-bit interface identifier to the prefix contained in the router advertisement.  Stateless DHCPv6: This is not mentioned as an option given in router advertisements [RFC2461]. Alcatel-Lucent Page 53 of 148...
  • Page 54 600; option dns-recursive-name-server 2001:db8::1; v6-dynamic-dhcp range 2001:db7::2 2001:db7::ff policy minimum-requested-lifetime 650; policy rapid-commit-enabled true; option dns-recursive-name-server 2001:db8::1; option domain-search-list example.com; /flash/switch/ dhcpdv6.pcy ;QIPOrgID=4 AbusiveClientMonitorPeriod=30 AbusiveClientWarningCount=30 AbusiveClientLockout=30 AddManualToGlobalDuidPool=1 AllowClientPacketsWithInvalidOptions=1 AllowUnencodedFqdn=1 ClientFqdnOptionSupport=client DefaultLease=6000 DHCPv6SocketAddr=2001:db8::1 RegisteredClientOnly=0 ReplyToUnmanagedInformationRequests=1 SendRequestedParamsOnly=1 Alcatel-Lucent Page 54 of 148...
  • Page 55 : 30000 ms Retransmission Interval : 1000 ms DAD Transmits Site Prefix Length : 64 Site Id Forwarding : disabled Advertising : disabled Neighbor Discovery : enabled Neighbor Unreachability Detection : enabled Router Discovery : enabled Alcatel-Lucent Page 55 of 148...

  • Page 56: Troubleshooting Udp Relay

    This packet flow description is described what happens when the UDP packets entered the OS10K/OS6900/OS6860/OS6860E unit. Also assume the end station is on VLAN 30 on VRF 2 with the ip interface for VLAN 30 is 10.30.0.254. Alcatel-Lucent Page 56 of 148...

  • Page 57: Dhcp Snooping Binding Database

    The physical port where the DHCP request is coming from;  The VLAN Id where the DHCP request packet is coming from;  The lease time of the IP Address;  The type/nature of how the binding entry is populated, either static or dynamic. Alcatel-Lucent Page 57 of 148...
  • Page 58 When the dhcpSnoopingBindingDatabasesyncTimeout is changed, the previous timer is stopped and the system will execute the timeout out with respect to the time that the timeout value is changed. (Start from fresh).  Ingress Source Filtering can only be enabled on the "client-only" ports. Alcatel-Lucent Page 58 of 148...

  • Page 59: Basic Troubleshooting

    -> no ip helper statistics -> no ip helper statistics global-only -> no ip helper statistics server-only Clear statistics in the standard mode: -> no ip helper statistics address <ipv4addr> Clear statistics in the per vlan mode: Alcatel-Lucent Page 59 of 148...
  • Page 60 15 level debug3 An example of UDP Relay related logs: -> show log swlog | grep -E "udpRelay|udprelay" <snap> swlogd: ipni udprelay debug2(7) port-chk: udp-dport 53 Alcatel-Lucent Page 60 of 148...

  • Page 61: Advanced Troubleshooting

    1 1 S UDP 0007e9:1ff566->e8e732:ab17bd IP 192.168.100.100->192.168.10.254 UDP 68,67 1 C S FLD e8e732:ab17bd->ffffff:ffffff ARP Request 192.168.10.253->192.168.10.254 1 1 R CMM e8e732:ab17bd->ffffff:ffffff ARP Request 192.168.10.253->192.168.10.254 1 1 S FLD e8e732:ab17bd->ffffff:ffffff ARP Request 192.168.10.253->192.168.10.254 1 1 R 1/1/1 00e0b1:f47b19->e8e732:ab17bd ARP Reply 192.168.10.254->192.168.10.253 Alcatel-Lucent Page 61 of 148...

  • Page 62: Troubleshooting In Maintenance Shell

    1 1 S 1/1/1 e8e732:ab17bd->00e0b1:f47b19 IP 192.168.10.253->192.168.10.254 UDP 67,67 9.5. Troubleshooting in Maintenance Shell Warning: Maintenance Shell commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel-Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.

  • Page 63: Troubleshooting Qos

    OS6900 Platform uses Trident in the data path which supports 10 slices. In 10 slices, first 4 slices contain 128 entries per slice and the remaining 6 slices have 256 entries per slice, resulting in total of 2048 IFP entries. Reserved slices OS6860 Helix4 has 16 slices of 256 entries each: Alcatel-Lucent Page 63 of 148...
  • Page 64 ACLs are a specific type of QoS policy used for Layer 2, Layer 3/4, and multicast filtering. The below is intended to provide the QoS/ACL necessary information on the OmniSwitch 6900 series to successfully configure and design a QoS/ACL policy in your networks. Alcatel-Lucent Page 64 of 148...
  • Page 65 Layer 4 None 802.1p only IP Multicast(IGMP) Destination Only Destination Only Destination Only None Policy Action Combination Matrix & Drop Priority Stamp/Map Maximum Bandwidth Redirect Port Redirect Linkagg Drop Priority Stamp/Map Maximum Bandwidth Redirect Port Alcatel-Lucent Page 65 of 148...
  • Page 66 Change in precedence will automatically revisit all the slices and TCAM rule allocation.  Efficient usage of the policy rule precedence allows the user to configure more rules and can avoid reaching the system limitation.  Policy Types can be summarized. TCAM allocation rules and practical examples Alcatel-Lucent Page 66 of 148...
  • Page 67 256/254 256/255 256/256 256/256 1/(0) 256/256 256/256 256/256 256/256 256/256 256/256 256/256 256/256 256/256 256/256 256/256 256/256 Mixing single source IP and destination IP in a policy condition will consume only one TCAM entry Configuration Alcatel-Lucent Page 67 of 148...
  • Page 68 256/256 256/256 256/256 256/256 256/256 Combining different policy groups in the policy rules can consume one TCAM rule for every possible combination of match between the groups Configuration policy network group g1 1.1.1.1 2.2.2.2 3.3.3.3 Alcatel-Lucent Page 68 of 148...
  • Page 69 Type Total/Free CAM Total/Free Total/Free Total/Free 1/1/(0) 32/32 128/128 128/128 128/128 128/127 128/127 128/128 128/125 128/125 128/128 128/127 128/127 128/128 256/256 256/256 256/256 256/256 256/256 256/256 256/255 256/255 256/256 256/255 256/256 256/256 256/255 256/254 256/254 Alcatel-Lucent Page 69 of 148...
  • Page 70 IPv6 rules and destination IPv6 rules forces the system to program each entry in different slice pair because the user is letting the system using the default precedence order. Warning: The rule precedence is based on the order in which the rule entry is entered or by defining the Alcatel-Lucent Page 70 of 148...
  • Page 71 2 previous examples were consuming too many slices and were not supported on the system. -> show qos slice Slot/ Ranges Rules Counters Meters Unit Type Total/Free CAM Total/Free Total/Free Total/Free 1/1/(0) 32/32 128/128 128/128 128/128 128/127 128/127 128/128 128/125 128/125 128/128 Alcatel-Lucent Page 71 of 148...
  • Page 72 11111111 11111110 (mask port 2-3) One additional TCAM rule is used to match when the port number equals to 4: TCAM rule 2 value2 00000000 00000100 (match port 4) mask2 11111111 11111111 (mask port 4) Alcatel-Lucent Page 72 of 148...
  • Page 73 Another rule is used to match when the port number is equal 2: TCAM rule 2 value2 00000000 00000010 (match port 2) mask2 11111111 11111111 (full mask) Example 3: Layer 4 – a port range Alcatel-Lucent Page 73 of 148...
  • Page 74 When the policy rule is consuming more than 5 regular TCAM rules the system automatically allocates a dedicate hardware TCP/UDP range for this policy so that it only consumes 1 TCAM rule, this allows the system to save TCAM rules for other rules. -> show qos slice Alcatel-Lucent Page 74 of 148...

  • Page 75: Basic Troubleshooting

    To determine how many rules and masks are being used by the system use "show qos slice" "show commands (available from the CLI). qos slice slot/port"  In case of heavy traffic matching a rule with enable, it is possible to observe packet loss "log" Alcatel-Lucent Page 75 of 148...

  • Page 76: Advanced Troubleshooting

    - 1638 576[0 IPMS_IGMP( 50) 0 - 1894 576[0 IPMS_V4Control( 51) 0 - 1540 - 1639 1829[5 IPMS_V4Control( 51) 0 - 1895 1829[0 IPMS_V4Data( 52) 0 8 1541 - 1640 IPMS_V4Data( 52) 0 - 1896 Alcatel-Lucent Page 76 of 148...
  • Page 77 Entry U Slice CIDU CIDL MIDU MIDL TCAM Count[+] Green[+] Red[+] NotGreen[+] List 2: 6 entries set up IPMS_MLD( 78) 0 IPMS_V6Control( 81) 0 IPMS_V6Data( 82) 0 IPMS_V6Resolved( 83) 0 MPLSTrust( 124) 0 MIMTrust( 144) 0 List 7: Alcatel-Lucent Page 77 of 148...
  • Page 78 1838[1838 vrrp( 65) 0 icmp( 66) 0 rip( 67) 0 bgpsrc( 68) 0 1766[1766 bgpdst( 69) 0 bfdecho( 70) 0 bfdreply( 71) 0 telnet( 72) 0 ssh( 73) 0 http( 74) 0 snmp( 75) 0 Alcatel-Lucent Page 78 of 148...

  • Page 79: Troubleshooting In The Maintenance Shell

    183182[183182 10.4. Troubleshooting in the Maintenance Shell Warning: Maintenance Shell commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel-Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.
  • Page 80 Display rules configured in TCAM (the entry number corresponds to the TCAM column from << debug qos internal >> output): "slot 1 list 255 verbose" BCM.0> d chg fp_policy_table FP_POLICY_TABLE.ipipe0[256]: <COUNTER_MODE=0x38,COUNTER_INDEX=2> FP_POLICY_TABLE.ipipe0[257]: <EVEN_PARITY=1,COUNTER_MODE=7,COUNTER_INDEX=3> FP_POLICY_TABLE.ipipe0[258]: <EVEN_PARITY=1,CPU_COS=0xe,COUNTER_MODE=0x38,CHANGE_CPU_COS=1> FP_POLICY_TABLE.ipipe0[259]: <G_COPY_TO_CPU=1,EVEN_PARITY=1,COUNTER_MODE=7,COUNTER_INDEX=1> FP_POLICY_TABLE.ipipe0[260]: <Y_COPY_TO_CPU=3,R_COPY_TO_CPU=3,G_COPY_TO_CPU=3,COUNTER_MODE=0x38,COUNTER_INDEX=1> FP_POLICY_TABLE.ipipe0[261]: <Y_COPY_TO_CPU=3,R_COPY_TO_CPU=3,G_COPY_TO_CPU=3,COUNTER_MODE=7,COUNTER_INDEX=2> FP_POLICY_TABLE.ipipe0[263]: <Y_CHANGE_PKT_PRI=5,Y_CHANGE_D SCP=3,Y_CHANGE_COS_OR_INT_PRI=5,R_CHANGE_PKT_PRI=5, R_CHANGE_D SCP=3,R_CHANGE_COS_OR_INT_PRI=5,G_CHANGE_PKT_PRI=5,G_CHANGE_DSCP_TOS=3,G_CHANGE_COS_OR_INT_PRI=5> Alcatel-Lucent Page 80 of 148...
  • Page 81 FP_METER_TABLE.ipipe0[2]: <BUCKETSIZE=1,BUCKETCOUNT=0xffff> FP_METER_TABLE.ipipe0[3]: <BUCKETSIZE=1,BUCKETCOUNT=0x3fffffff> Display counter configured in TCAM (the entry number does NOT correspond to the TCAM column from, it corresponds to counter configured in FP_POLICY_TABLE): BCM.0> d chg fp_counter_table FP_COUNTER_TABLE.ipipe0[399]: <PACKET_COUNTER=0x1bd48e,BYTE_COUNTER=0x1ff1f2b9> FP_COUNTER_TABLE.ipipe0[1548]: <PACKET_COUNTER=0x1a5e,BYTE_COUNTER=0xd2f00> Alcatel-Lucent Page 81 of 148...

  • Page 82: Troubleshooting Rip

    Route Map: local_map_1 Sequence Number: 1 Action permit match ip-address 102.0.0.0/8 redist-control all-subnets deny set metric 1 effect none Verify that RIP is enabled globally and redistribution is also enabled, using the show ip rip command. Alcatel-Lucent Page 82 of 148...
  • Page 83 FILEID: fileName[/flash/swlog1.log], endPtr[1433], configSize[500000], mode[1] Time Stamp Application Level Log Message ------------------------+--------------+-------+-------------------------------- TUE JUN 03 14:23:53 2008 SYSTEM info Switch Logging cleared by command. File Size=1000000 bytes TUE JUN 03 14:24:00 2008 info tRip::ripRecv:Received packet from 102.102.0.26 Alcatel-Lucent Page 83 of 148...
  • Page 84 TUE JUN 03 14:24:00 2008 info es=25 len=504 TUE JUN 03 14:24:00 2008 info tRip::ripRecv:Received packet from 102.102.0.26 TUE JUN 03 14:24:00 2008 info tRip::ripRecv: Rx: RESP ver=v2 src=102.102.0.26 inIf=102.102.0.122 port=520 tupl Alcatel-Lucent Page 84 of 148...

  • Page 85: Troubleshooting Ospf

    According to current AOS architecture, the control/management functions are performed by CMM and data forwarding functions are performed by NI. OSPF task runs in control plane building the forwarding information which will be used by the data plane for data forwarding. OSPF task functions in centralized mode Alcatel-Lucent Page 85 of 148...
  • Page 86 Graceful LSA Update message should be sent out first before the Hello packet to the neighbor, to avoid adjacency reset due to OSPF state mismatch in Hello packet. Sub-second convergence during Virtual Chassis Takeover To achieve sub-second convergence during VC takeover, the requirements for OSPF graceful restart should be met. Alcatel-Lucent Page 86 of 148...

  • Page 87: Minimum Working Configuration

    Time (in seconds) = 5, MTU Checking = Disabled, # of Routes = 1, # of AS-External LSAs = 0, # of self-originated LSAs = 1, # of LSAs received = 2, External LSDB Limit = -1, Alcatel-Lucent Page 87 of 148...
  • Page 88 Area Id = 0.0.0.0, Designated Router IP Address = 192.168.1.2, Designated Router RouterId = 192.168.1.2, Backup Designated Router IP Address = 192.168.1.1, Backup Designated Router RouterId = 192.168.1.1, (bytes) = 1500, Metric Cost = 1, Alcatel-Lucent Page 88 of 148...

  • Page 89: Advanced Troubleshooting

    "vrfid" corresponds to a given non-default VRF name, one must examine the "ps" output in the su shell. An example output from a switch that is running OSPF in three VRFs: -> show vrf Virtual Routers Profile Protocols --------------------+-------+------------------- Alcatel-Lucent Page 89 of 148...
  • Page 90 Messages for OSPF’s initialization setup only. setup Messages for OSPF’s time related events only. Timers are set for interfaces and LSAs. time Messages for MIP processing of OSPF specific commands only. Messages for OSPF’s Task Manager communication events only. Alcatel-Lucent Page 90 of 148...
  • Page 91 For more refined control, one can tweak the internal log levels with a "debug" cli command. Internally, the ospf log levels are identical to the "drclog" logging levels that may be familiar to users of AOS 6 releases:  level = 1 => info  level = 50 => errors Alcatel-Lucent Page 91 of 148...
  • Page 92 Part No.032996-00 Rev.A AOS Release 7.X and 8.X January 2015  level = 60 => informative  level = 75 => detailed  level = 255 => all Example: debug ip ospf set lsdb 100 Alcatel-Lucent Page 92 of 148...

  • Page 93: Troubleshooting Bgp

    Build the Neighbor - Troubleshooting Commands show ip bgp !global settings show ip bgp statistics !global statistics !address, AS, state, … show ip bgp neighbors CHECKS: ip addresses, ASN, Router-ID, MD5 show ip bgp neighbor <ip-addr> !per neighbor details Alcatel-Lucent Page 93 of 148...
  • Page 94 <m1> 90 admin-state enable !enable ip bgp neighbor <ip_addr> route-map <m1> filterout |in !check <ip_addr> route-map <m1> filter-in ip bgp neighbor clear soft [in | out] !re-config show ip route-map Border Router Redistribution Policies Alcatel-Lucent Page 94 of 148...
  • Page 95 //inspect the available paths show ip route protocol [bgp | ospf…] //aka Cisco sh ip bgp default route preference //select the routes based on lowest IPv4 route preference (adminstrative dist) static ospf isis isisl2 ebgp ibgp import Alcatel-Lucent Page 95 of 148...
  • Page 96 "v6if-v99" vlan 99 ipv6 address 2001:470:4979:99::1/64 "v6if-v99" ipv6 bgp neighbor 2001:470:4979:99::24 ipv6 bgp neighbor 2001:470:4979:99::24 timers 10 30 ipv6 bgp neighbor 2001:470:4979:99::24 remote-as 2152 ipv6 bgp neighbor 2001:470:4979:99::24 activate-ipv6 ipv6 bgp neighbor 2001:470:4979:99::24 status enable Alcatel-Lucent Page 96 of 148...

  • Page 97: Advance Troubleshooting

    Nov 13 22:27:20 (none) swlogd: bgp_0 fsm debug2(7) [peer(192.168.10.1),65555] restarting hold timer [90] sec Nov 13 22:27:20 (none) swlogd: bgp_0 fsm debug1(6) vrfId 0: [peer(192.168.10.1),65555] EXIT: EVENT keepalive_msg_recv_event [NEXT_STATE Established] Nov 13 22:27:23 (none) swlogd: bgp_0 info debug3(8) [peer(192.168.10.1),65555] START at Time 02h:42m:08s:000ms Alcatel-Lucent Page 97 of 148...
  • Page 98 Nov 13 22:27:35 (none) swlogd: bgp_0 peer debug3(8) vrfId 0: [peer(192.168.10.1),65555] Added route advrtsmt timer for 26s Nov 13 22:27:35 (none) swlogd: bgp_0 info debug3(8) [peer(192.168.10.1),65555] START at Time 02h:42m:20s:000ms Nov 13 22:27:35 (none) swlogd: bgp_0 info debug3(8) [peer(192.168.10.1),65555] END at Time 02h:42m:20s:000ms Alcatel-Lucent Page 98 of 148...

  • Page 99: Troubleshooting Ip Multicast Switching (Ipms)

    $(pidof ipmscmm) 'p ipms::bcm_max' getreg MC_CONTROL_5 d chg l2mc d l3_ipmc d l3_entry_2 d l3_ipmc _______________________________________________________________ 14.1. Introduction OS6900 and OS10K Hardware Limitations Egress ASICs Platforms Ingress Limits Limits OS10K L3_IPV4_MULTICAST L3_IPV6_MULTICAST L3_IPMC OS10K-GNI-C48E OS10K-GNI-U48E OS10K-XNI-U32S OS10K-XNI-U32S OS6900 Alcatel-Lucent Page 99 of 148...
  • Page 100 IP: Type of Service = 192 (0xC0) IP: Total Length = 32 (0x0020) IP: Identification = 4229 (0x1085) IP: Fragment Offset IP: Time to Live = 1 (0x01) IP: Protocol = IGMP IP: Checksum = 0x… Alcatel-Lucent Page 100 of 148...
  • Page 101 When the spoofing feature is enabled, source IP address of each IGMP Membership Report is updated according to IP interface address in the VLAN in which the IGMP Membership Report was received. Other IGMP messages are not affected. Example (without IPMVLAN): Alcatel-Lucent Page 101 of 148...
  • Page 102 If this feature is enabled on a querier, then all IGMP Membership queries are forwarded towards the streaming device. Neighbor The "neighbor" concept was introduced for IGMPv3 in RFC 3376 . This concept is also used for IGMPv2. The extract from the RFC 3376 Alcatel-Lucent Page 102 of 148...
  • Page 103 = 04 (0x04) IP: Header Length = 20 (0x14) IP: Differentiated Services Field = 0 (0x00) IP: Type of Service = 00 (0x00) IP: Total Length = 28 (0x001C) IP: Identification = 0 (0x0000) IP: Fragment Offset Alcatel-Lucent Page 103 of 148...
  • Page 104 Note: IGMP Membership Report type 3 is used for registration of new subscribers. One message can be used to register multiple multicast groups. IGMP Membership Report in IGMP version 3 is always sent with 224.0.0.22 destination IP address. MAC: ------ MAC Header ------ MAC: Alcatel-Lucent Page 104 of 148...
  • Page 105 IGMP Header ----------- IGMP: IGMP: Version (0x3) IGMP: Type = Membership Report v3 (0x22) IGMP: Reserved (0x0) IGMP: Checksum 0xE7F0 IGMP: Reserved (0x0) IGMP: Number of Group Records = (0x1) IGMP: ------ IGMP Group Record ----------- Alcatel-Lucent Page 105 of 148...

  • Page 106: Basic Troubleshooting

    = disabled, Spoofing = disabled, Zapping = disabled, Querier Forwarding = disabled, Flood Unknown = disabled, Version = 2, Robustness = 2, Query Interval (seconds) = 125, Query Response Interval (tenths of seconds) = 100, Alcatel-Lucent Page 106 of 148...
  • Page 107 00-00-00-00-00-01 modid 0 devport 48 <snap> swlogd: ipmsCmm msg debug1(6) cni recv len 76 <snap> swlogd: ipmsCmm msg debug1(6) cni report vlan 1 stack 0 port 1/1/48 svp 0 host 192.168.1.1 sa 00-00-00-00-00-01 modid 0 devport 48 Alcatel-Lucent Page 107 of 148...
  • Page 108 0.0.0.0 orig 0.0.0.0 index 3 <snap> swlogd: ipmsCmm msg debug1(6) sec chas 0 flow vlan 1 group 239.0.0.1 host 10.0.0.1 dest 0.0.0.0 orig 0.0.0.0 encap 1 index 3 <snap> swlogd: ipmsCmm mip debug1(6) process Alcatel-Lucent Page 108 of 148...

  • Page 109: Advanced Troubleshooting

    A more detailed version of "show ip multicast group" -> debug ip multicast member Total 1 Members Group Address/ VLAN Port Mode Count Life Query Count Source Address ---------------+-----+---------+--------+------+-----+------+------+---+-----+----- 239.0.0.1 1/1/48 exclude A more detailed version of "show ip multicast flow" Alcatel-Lucent Page 109 of 148...

  • Page 110: Troubleshooting In The Maintenance Shell

    0 | V3 Queries 14.4. Troubleshooting in the Maintenance Shell Warning: Maintenance Shell commands should only be used by Alcatel-Lucent personnel or under the direction of Alcatel-Lucent. Misuse or failure to follow procedures that use Maintenance Shell commands in this guide correctly can cause lengthy network down time and/or permanent damage to hardware.

  • Page 111: Troubleshooting In Bshell

    BCM.0> d l3_ipmc L3_IPMC.ipipe0[0]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0,L2_BITMAP_W0=0,L2_BITMAP=0x0000000000000000,HIGIG_TRUNK_OVERRIDE_PROFILE_PTR=0,EVE N_PARITY=1> L3_IPMC.ipipe0[1]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0xffffff,L2_BITMAP_W0=0xfffffffe,L2_BITMAP=0x00fffffffffffffe,HIGIG_TRUNK_OVERRIDE_P ROFILE_PTR=0,EVEN_PARITY=0> L3_IPMC.ipipe0[2]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0,L2_BITMAP_W0=1,L2_BITMAP=0x0000000000000001,HIGIG_TRUNK_OVERRIDE_PROFILE_PTR=0,EVE N_PARITY=0> L3_IPMC.ipipe0[3]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0xc00000,L2_BITMAP_W0=4,L2_BITMAP=0x00c0000000000004,HIGIG_TRUNK_OVERRIDE_PROFILE_PT R=0,EVEN_PARITY=0> A new entry is added to this table when a new IPMS forwarding entry is created (an example): Alcatel-Lucent Page 111 of 148...
  • Page 112 L2_BITMAP_W1=0,L2_BITMAP_W0=0,L2_BITMAP=0x0000000000000000,HIGIG_TRUNK_OVERRIDE_PROFILE_PTR=0,EVE N_PARITY=1> L3_IPMC.ipipe0[1]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0xffffff,L2_BITMAP_W0=0xfffffffe,L2_BITMAP=0x00fffffffffffffe,HIGIG_TRUNK_OVERRIDE_P ROFILE_PTR=0,EVEN_PARITY=0> L3_IPMC.ipipe0[2]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0,L2_BITMAP_W0=1,L2_BITMAP=0x0000000000000001,HIGIG_TRUNK_OVERRIDE_PROFILE_PTR=0,EVE N_PARITY=0> A new entry is added to this table when a new IPMS forwarding entry is creatred (an example): BCM.0> d l3_ipmc L3_IPMC.ipipe0[3]: <VALID=1,REMOVE_SGLP_FROM_L3_BITMAP=0,L3_BITMAP_W1=0,L3_BITMAP_W0=0,L3_BITMAP=0x0000000000000000, L2_BITMAP_W1=0xc00000,L2_BITMAP_W0=0x1000,L2_BITMAP=0x00c0000000001000,HIGIG_TRUNK_OVERRIDE_PROFI LE_PTR=0,EVEN_PARITY=0> Alcatel-Lucent Page 112 of 148...

  • Page 113: Troubleshooting Ip Multicast Routing (Ipmr)

    "vlan20" ip pim interface "Loopback0" ip pim static-rp 224.0.0.0/4 192.168.254.1 ip pim sparse admin-state enable ip pim dense admin-state disable ipv6 pim sparse admin-state disable ipv6 pim dense admin-state disable ! DVMRP: ! IPMR: Alcatel-Lucent Page 113 of 148...

  • Page 114: Basic Troubleshooting

    00h:15m:19s 192.168.1.19 239.0.0.4 vlan10 00h:15m:19s 192.168.1.19 239.0.0.5 vlan10 00h:15m:19s -> show ip pim groute | head Total 100 (*,G) Group Address RP Address RPF Interface Upstream Neighbor UpTime Mode ---------------+---------------+--------------------------------+-----------------+-----------+-- ---- 239.0.0.1 192.168.254.1 00h:28m:33s Alcatel-Lucent Page 114 of 148...

  • Page 115: Advanced Troubleshooting

    1,001/s T127.xe11 TPOK.xe11 224,509 +1,184 1,002/s TPKT.xe11 224,509 +1,184 1,002/s TMCA.xe11 224,497 +1,184 1,002/s TBYT.xe11 14,375,022 +75,780 64,106/s BCM.0> show c xe11 PERQ_PKT(0).xe11 223,494 +1,182 1,000/s PERQ_BYTE(0).xe11 14,303,616 +75,648 63,994/s UC_PERQ_PKT(9).xe11 1,003 UC_PERQ_BYTE(9).xe11 69,738 58/s Alcatel-Lucent Page 115 of 148...

  • Page 116: Troubleshooting 802.1X

    _____________________________________________________________ This section concerns the OmniSwitch 6860 running AOS 8 1) Verify the configuration as there are multiple profiles and associations to create: RADIUS server to aaa profile: aaa radius-server "clearpass" host 172.26.61.6 aaa profile "clearpass-aaa-profile"...
  • Page 117 On the uplink to the server check the exchange between the switch and the RADIUS server. If you can ping the server but the switch get not reply, you may have a firewall issue, RADIUS uses ports 1812 and 1813 by default. Alcatel-Lucent Page 117 of 148...

  • Page 118: Troubleshooting Universal Network Profiles (Unp)

    4IPMC_L2_ENABLE=0,V4IPMC_ENABLE=0, USE_PORT_TABLE_GROUP_ID=0,USE_IVID_AS_OVID=0,USE_INNER_PRI=0,URPF_MODE=0,URPF_DEFAULTROUTECHECK=0 ,TX_DEST_PORT_ENABLE=0,TX_DEST_PORT=0, TRUST_INCOMING_VID=1,TRUST_DSCP_V6=0,TRUST_DSCP_V4=0,TRUST_DOT1P_PTR=0,TRILL_ENABLE=0,TAG_ACTION_ PROFILE_PTR=3,SUBNET_BASED_VID_ENABLE=0, RTAG7_PORT_LBN=0,RTAG7_HASH_CFG_SEL_TRUNK=0,RTAG7_HASH_CFG_SEL_TRILL_ECMP=0,RTAG7_HASH_CFG_SEL_LB ID=0,RTAG7_HASH_CFG_SEL_HIGIG_TRUNK=0, RTAG7_HASH_CFG_SEL_ECMP=0,RESERVED_1=0,RESERVED_0=0,REMOVE_HG_HDR_SRC_PORT=0,REMOTE_CPU_EN=0,PVLA N_ENABLE=0,PROTOCOL_PKT_INDEX=2, PROHIBITED_DOT1P=0,PRI_MAPPING=0xfac688,PORT_VID=1,PORT_TYPE=0,PORT_PRI=0,PORT_OPERATION=0,PORT_D IS_UNTAG=0,PORT_DIS_TAG=0, //snip// Modified the parameter of PORT_VID in BCM. After this changes, user entry on port 1 is updated correctly (as same as port 19) Alcatel-Lucent Page 118 of 148...
  • Page 119 BCM.0> mod port 1 1 PORT_VID=1 OS6900-3> show unp user User Learning Port Username Mac address Vlan Status Source ------+-----------------+-----------------+---------------+----+-------------------------------- +------+---------- 1/1/1 00:13:72:7a:f3:73 00:13:72:7a:f3:73 192.168.1.253 4095 spb1 Active Local 1/1/19 00:13:72:7a:11:11 00:13:72:7a:11:11 12.13.14.15 4095 spb2 Active Local Alcatel-Lucent Page 119 of 148...

  • Page 120: Troubleshooting Snmp

    The OmniSwitch supports SNMP v1, v2, and v3. The most often mistake is when the wrong workstation IP address is configured. The workstation can ping the switch, but no traps are being received. To verify the SNMP Management Station, use the show snmp station command: Alcatel-Lucent Page 120 of 148...
  • Page 121 15 seconds 18 esmDrvTrapDropsLink interface 15 seconds 19 portViolationTrap interface 15 seconds 20 dvmrpNeighborLoss ipmr 15 seconds 21 dvmrpNeighborNotPruning ipmr 15 seconds 22 risingAlarm rmon 15 seconds 23 fallingAlarm rmon 15 seconds 24 stpNewRoot 15 seconds Alcatel-Lucent Page 121 of 148...
  • Page 122 99 vRtrIsisAreaMismatch isis 15 seconds 100 vRtrIsisRejectedAdjacency isis 15 seconds 101 vRtrIsisLSPTooLargeToPropagate isis 15 seconds 102 vRtrIsisOrigLSPBufSizeMismatch isis 15 seconds 103 vRtrIsisProtoSuppMismatch isis 15 seconds 104 vRtrIsisAdjacencyChange isis 15 seconds 105 vRtrIsisCircIdExhausted isis 15 seconds Alcatel-Lucent Page 122 of 148...
  • Page 123 15 seconds 180 pethMainPowerUsageOnNotification module 15 seconds 181 pethMainPowerUsageOffNotification module 15 seconds 182 chasTrapsBPSFwUpgradeAlert chassis 15 seconds 183 alaAppMonAppRecordFileCreated app-mon 15 seconds 184 alaAppMonFlowRecordFileCreated app-mon 15 seconds 185 alaDPIFlowRecordFileCreated 15 seconds 186 alaLbdStateChangeToShutdown 15 seconds Alcatel-Lucent Page 123 of 148...

  • Page 124: Snmp Security

    By default, the switch is set to privacy all, which means the switch accepts only authenticated and encrypted v3 Sets, Gets, and Get-Nexts.To verify the SNMP security setting, use the show snmp security command: -> show snmp security snmp security = no security Alcatel-Lucent Page 124 of 148...

  • Page 125: Snmp Statistics

    NMS station might be listening to some other port for the traps. Make sure that the switch matches with the NMS's port setup using the show snmp station command. For example: Alcatel-Lucent Page 125 of 148...
  • Page 126 Use the debug trap counts command to verify the list of traps generated: -> debug trap counts +------------------+ Trap Manager +------------------+ Trap absorbed dropped [ 0] [ 1] [ 2] [ 3] [ 4] [ 5] [ 8] [24] [25] [27] [120] [121] [183] Total Alcatel-Lucent Page 126 of 148...

  • Page 127: Troubleshooting Power Over Ethernet

    Default behaviour: PoE is disabled by default PoE status command: -> show lanpower slot <chassis/slot> Below is an example: -> show lanpower slot 1/1 Port Maximum(mW) Actual Used(mW) Status Priority On/Off Class Type ----+-----------+---------------+-----------+---------+--------+-------+---------- 30000 Searching Alcatel-Lucent Page 127 of 148...
  • Page 128 This feature is enabled by default. Priority disconnect is used by the system software in determining whether an incoming PD will be granted or denied power when there are too few watts remaining in the PoE power budget for an additional device. -> show lanpower slot <chassis/slot> priority-disconnect Alcatel-Lucent Page 128 of 148...
  • Page 129 (i.e., critical) priority. In the event of a power management issue, inline power to critical is maintained as long as possible. Below is an example: -> show lanpower slot 1/1 Port Maximum(mW) Actual Used(mW) Status Priority On/Off Class Type ----+-----------+---------------+-----------+---------+--------+-------+---------- 30000 Searching 30000 Searching 30000 Searching …. Alcatel-Lucent Page 129 of 148...

  • Page 130: Troubleshooting Ethernet Ring Protection (Erp)

    "slot 1 list 1 verbose" | grep ERP ___________________________________________________________________ 20.1. Troubleshooting ERP on OmniSwitch ERP is disabled by default. Below is the command to verify the overall ERP status: -> show erp Below is an example (default): Alcatel-Lucent Page 130 of 148...
  • Page 131 -> show erp ring 1 Ring Id : 1, Ring Type : Normal Ring, Ring Port1 : 1/1/1, Ring Port2 : 1/1/2, Ring Status : enabled, Service VLAN : 1001, Revertive Mode : enable, WTR Timer (min) : 5, Alcatel-Lucent Page 131 of 148...
  • Page 132 ERP Protected VLAN is the VLAN that is added to the ERP ring. ERP determines the forwarding state of protected VLANs. Filtering Database (FDB) is a database that stores filtered data according to the R-APS messages recieved. This database also maintains an association table that identifies the master rings for a given sub-ring. Alcatel-Lucent Page 132 of 148...
  • Page 133 DNF) is transmitted immediately from the other ring port. The node then unblocks its blocked port and blocks the down port. It is important to block the port which is going down so that when the port comes up it will be Alcatel-Lucent Page 133 of 148...
  • Page 134 No Request RPL Block PDUs Sent : 3, Recv : 1170, Drop : 0 Invalid R-APS PDUs Recv : 0 Ring Port : 1/1/2 Signal Fail PDUs Sent : 0, Recv : 0, Drop : 0 No Request PDUs Alcatel-Lucent Page 134 of 148...
  • Page 135 Ethoam Event : disabled, Remote-endpoint Id : none Troubleshooting Commands Commands will be provided accordingly from Support/Engineering, on a "case-to-case" basis. Debug command -> debug qos internal "slot 1 list 1 verbose" | grep ERP Alcatel-Lucent Page 135 of 148...

  • Page 136: Troubleshooting Shortest Path Bridging (Spb)

    Service Access Points associated with this SPB service. Mcast mode provide the multicast replication mode (Headend or Tandem) for the service. For example: -> show service spb Legend: * denotes a dynamic object SPB Service Info Alcatel-Lucent Page 136 of 148...
  • Page 137 Stats : N, : 9194, VlanXlation : N, ISID : 13000, BVlan : 4003, MCast-Mode : Headend, Tx/Rx : 0/0 Sap Trusted:Priority/ Sap Description / Identifier Oper Stats Sdp SystemId:BVlan Intf Sdp SystemName ----------------------+----+----+-----+--------------------+--------+---------------------------- ---- Alcatel-Lucent Page 137 of 148...

  • Page 138: Spb Debug Information

    The command show service l2profile display the Layer 2 profile configuration information for the bridge. This type of profile is applied to access (customer-facing) ports and specifies how to process Layer 2 protocol frames ingressing on such ports. -> show service l2profile Alcatel-Lucent Page 138 of 148...

  • Page 139: Advanced Troubleshooting Scenarios

    From this result, it shows isid 4001 bound to bvlan 4001 and it is using SDP 32786 Do “show service sdp” and have the resul iii. 32786 00e0.b1e7.0bd3:4001 Up Up SPB It shows the MAC address 00e0.b1e7.0bd3 is the system-id of the remote switch at the other end of the SDP. Alcatel-Lucent Page 139 of 148...

  • Page 140: Bshell Troubleshooting

    When a SAP is disabled, the virtual port still exists in hardware as well as in software. But in software we configure that virtual port down and configure hardware to set CML flag as 1 which is to drop the packet received in this virtual port. For example: BCM.0> d chg source_vp SOURCE_VP.ipipe0[1]: <VFI=1,TPID_ENABLE=1,SD_TAG_MODE=1,EXP_PVLAN_VID=1,EVEN_PARITY=1,ENTRY_TYPE=1,DVP=1,DISABLE_VLAN_ CHECKS=1,DEFAULT_VLAN_TAG=1,DEFAULT_VID=1,CML_FLAGS_NEW=1,CML_FLAGS_MOVE=1,> Alcatel-Lucent Page 140 of 148...

  • Page 141: Troubleshooting Sflow

    "debug sflow show rate" will show the actual rate at the time the command is executed. sFlow is not designed to sample at a rate of 1:1. The recommended sample rates are: 10mbps = 200 100mbps = 500 1,000mbps = 1000 10,000mbps = 2000 Alcatel-Lucent Page 141 of 148...

  • Page 142: Sflow Debug

    1 receiver 1 port 1/38 interval 5 sflow poller 1 receiver 1 port 1/39 interval 5 sflow poller 1 receiver 1 port 1/40 interval 5 22.1. sFLOW Debug CLI Command syntax: DEBUG SFLOW DUMP STATISTICS Dumps statistics on the datagrams transferred. Alcatel-Lucent Page 142 of 148...

  • Page 143: Troubleshooting Port Mirroring And Port Monitoring

    2/1/14 as a destination port. # port-mirroring 1 no source 2/1/18 ERROR: Session 1 is enabled. Cannot be modified. *** disable port-mirroring 1 before you can remove any source port. port-mirroring 1 disable Alcatel-Lucent Page 143 of 148...

  • Page 144: Troubleshooting Port Monitoring

    ERROR: Exceeds Max Number of Sessions # show configuration snapshot pmm ! Port Mirroring: port-monitoring 3 source 1/1/14 file /flash/portmon2.cap size 1 timeout 0 bidirectional capture-type brief port-mirroring 1 destination 2/1/14 port-mirroring 2 destination 2/1/17 Alcatel-Lucent Page 144 of 148...
  • Page 145 To verify the status of the port monitoring session, the following command can be issued: # show port-monitoring status Sess Mon. Mon. Over Oper. Admin Capt. Max. File write Stat Stat Type Size Name -----+-------+----+-----+------+------+-------+------+----------------------- 1/1/19 Full 2048K /flash/pm3.cap Alcatel-Lucent Page 145 of 148...

  • Page 146: Troubleshooting Ipv6

    Legend: + indicates routes in-use Total IPRM IPv6 routes: 2 Destination/Prefix Gateway Address Interface Protocol Metric ---------------------------------------------------+---------------------------------------------+------------- + ::1/128 loopback LOCAL + 2001:db8:4100:1000::/64 fe80::eae7:32ff:feae:7811 v6if-v200 LOCAL Inactive Static Routes: Vlan Destination/Prefix Gateway Address Metric -----+---------------------------------------------+---------------------------------------------+-------+----- ----- Alcatel-Lucent Page 146 of 148...

  • Page 147: Troubleshooting Dhcpv6 Relay

    > show ipv6 dhcp relay DHCPv6 Relay: Enabled When the relay interface and the relay destination are configured the output is below: > show ipv6 dhcp relay DHCPv6 Relay: Enabled Interface Relay Destination(s) Status ---------------------------+-----------------------------------------+-------- Alcatel-Lucent Page 147 of 148...

  • Page 148: Troubleshooting A 6To4 Tunnel

    ::/0 2002:d468:8a89::137 v6if-6to4-137 18h 47m 26s Static UGS 137:35:35::/64 fe80::2d0:95ff:fe12:f470 v6if-tunnel-137 18h 51m 55s Local UC 195:35::/64 fe80::2d0:95ff:fe12:f470 v6if-to-eagle 18h 51m 55s Local UC 212:95:5::/64 fe80::2d0:95ff:fe12:f470 smbif-5 18h 51m 55s Local UC 2002::/16 2002:d423:2323::35 v6if-6to4-137 18h 51m 55s Other U Alcatel-Lucent Page 148 of 148...

This manual is also suitable for:

Omniswitch 6900

Table of Contents