Page 2
The customer hereby agrees that the use, sale, licence or other distribution of the products for any such application without the prior written consent of Alcatel, shall be at the customer's sole risk. The customer hereby agrees to defend and hold Alcatel harmless from any claims for loss, cost, damage, expense or liability that may arise out of or in connection with the use, sale, licence or other distribution of the products in such applications.
Page 3
SAMPLE END USER LICENSE AGREEMENT 1. LICENSE 1.1 Subject to the terms and conditions of this Agreement, Alcatel grants to Customer and Customer accepts a non-exclusive, non-transferable license to use any software and related documentation provided by Alcatel pursuant to this Agreement ("Licensed Program") for Customer's own internal use, solely in...
Page 4
3.2 Alcatel may terminate this Agreement: (a) upon notice to Customer if any amount payable to Alcatel is not paid within thirty (30) days of the date on which payment is due; (b) if Customer becomes bankrupt, makes an assignment for the benefit of its creditors, or if its assets vest or become subject to the rights of any trustee, receiver or other administrator;...
Page 5
LICENSED PROGRAMS WILL NOT INFRINGE UPON ANY THIRD PARTY RIGHTS. 6.3 Alcatel shall defend and indemnify Customer in any action to the extent that it is based on a claim that the Licensed Program furnished by Alcatel infringes any patent, copyright, trade secret or other intellectual property right, provided that...
Page 6
8.4 The Licensed Program may contain freeware or shareware obtained by Alcatel from a third party source. No license fee has been paid by Alcatel for the inclusion of any such freeware or shareware, and no license fee is charged to Customer for its use.
Page 7
8.9 No term or provision of this Agreement shall be deemed waived and no breach excused unless such waiver or consent is in writing and signed by the party claimed to have waived or consented. The waiver by either party of any right hereunder, or of the failure to perform or of a breach by the other party, shall not be deemed to be a waiver of any other right hereunder or of any other breach or failure by such other party, whether of a similar nature or otherwise.
Preface About this document The 5620 SAM Troubleshooting Guide provides task-based procedures and user documentation to: • collect data to help resolve issues in the network and network management domains • identify the root cause and plan corrective action for: •...
Use the 5620 SAM Installation and Upgrade Guide to install the 5620 SAM database, server, and client software. • Use the 5620 SAM User Guide for information about using the client GUI to perform network management functions. • Use the 5620 SAM Parameter Guide for definitions, ranges, dependencies, and defaults for configurable parameters from the 5620 SAM client GUI.
Caution indicates that the described activity or situation Caution — may, or will, cause service interruption. Notes provides information that is, or may be, of special Note — interest. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 12
Preface Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Troubleshooting guidelines................1-4 Before you call support ..................1-5 2 — Troubleshooting using 5620 SAM 5620 SAM troubleshooting process..............2-2 Troubleshooting the network................2-2 Troubleshooting network management............2-3 Alcatel 5620 Service Aware Manager, Release 2.1 R2 xiii May 2005 95-5887-01-00-B...
Page 14
Troubleshooting tools ..................2-3 OAM diagnostics.................... 2-3 Event log and property files ................2-4 Procedure 2-1 To collect troubleshooting logs and property files ....2-4 Workflow to troubleshoot your network using 5620 SAM ........2-5 Network troubleshooting 3 — Troubleshooting network alarms Troubleshooting using network alarms strategy ..........
Page 15
Procedure 7-3 Problem: There is excess disk activity on my Solaris platform ....................7-6 Procedure 7-4 Problem: There is not enough swap space added or the Solaris platform is disk bound ............7-8 Troubleshooting Windows platforms..............7-10 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 16
Procedure 8-8 Problem: Problem collecting large numbers of logged statistics records or other large queries ..........8-9 Procedure 8-9 Problem: Cannot view alarms from a 5620 SAM on a 5620 NM or 1354 BM ................8-10 Troubleshooting client GUI issues ..............8-11 Procedure 8-10 Problem: The GUI keeps shutting down ......
Page 17
Commitment of changes from a form and its sub-forms ......11-3 Service disruption warning................11-4 Duplicate configuration form conflicts ............11-5 11.2 Responding to 5620 SAM client GUI warning messages ........ 11-5 Procedure 11-1 To respond to a warning message........11-5 12 — Troubleshooting with Problems Encountered forms 12-1 12.1...
Page 18
Contents Glossary Index xviii Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Troubleshooting overview 1 — Troubleshooting process 2 — Troubleshooting using 5620 SAM Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 20
Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
1 — Troubleshooting process 1.1 Troubleshooting process 1.2 Troubleshooting problem-solving model 1.3 Troubleshooting guidelines 1.4 Before you call support Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
The maintenance of the baseline information is critical because a network is not a static environment. See the 5620 SAM Routine Maintenance Procedures Guide for more information on how to generate baseline information for 5620 SAM applications.
• Determine the network state before the problem appeared. • Extrapolate from network alarms and network events the cause of the symptoms. Try to reproduce the problem. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
1 — Troubleshooting process The following 5620 SAM features can help you identify the route cause of a problem: • alarms with vendor-specific and X.733 standardized probable causes • alarm history associated network conditions Plan corrective action and resolve the problem The corrective action required to resolve a problem depends on the problem type.
Collect the information listed in Table 1-1 before you call your TAC or technical support representative. The list of Alcatel support contacts is available from the Alcatel home page at www.alcatel.com. Click on the Support link. Table 1-1 Troubleshooting data collection for support...
Page 26
1 — Troubleshooting process Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
2 — Troubleshooting using 5620 SAM 2.1 5620 SAM troubleshooting process 2.2 Troubleshooting tools 2.3 Workflow to troubleshoot your network using 5620 SAM Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
The 5620 SAM converts SNMP traps from network devices to events and alarms. You can then use the 5620 SAM to correlate the events and alarms against the managed equipment, and the configured services and policies. A correlated event or alarm can cause fault conditions on multiple network objects and services.
• TCP/IP networking Troubleshooting tools The 5620 SAM supports the use of OAM diagnostic tools and event logs to help identify the root cause of a network or network management problem. OAM diagnostics The 5620 SAM supports configurable in-band and out-of-band, packet-based OAM diagnostic tools to troubleshoot your network service.
The number of log files generated can use large amounts of disk space if systems run for long periods with significant activity. Ensure that the contents of the various log directories are backed up on a regular basis. See the 5620 SAM Routine Maintenance Procedures Guide for more information about how to perform routine maintenance on your network.
Store the files in a secure location until they are sent to support, and ensure that the files are not overwritten. For example, if there are two 5620 SAM clients with troubleshooting issues, do not place the two nms-client.xml files in the same directory because one of them will be overwritten.
Page 32
Problem: Problem collecting large numbers of logged statistics records or other Procedure 8-8 large queries Problem: Cannot view alarms from a 5620 SAM on a 5620 NM or 1354 BM Procedure 8-9 (1 of 3) Alcatel 5620 Service Aware Manager, Release 2.1 R2...
Page 33
Procedure 9-3 Problem: Unsure of the status of my server Procedure 9-4 Problem: All SNMP traps from 7750 SRs are arriving at one 5620 SAM server, or Procedure 9-5 no SNMP traps are arriving Problem: Cannot discover more than one device or a resynchronization of devices...
Page 34
To identify the database activity for a user request Procedure 13-2 To identify the deployment results for a user request Procedure 13-3 To retrieve historical user logs Procedure 13-4 (3 of 3) Verify the solution. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
View related object information if the root cause is not found on the affected object. See Procedure 3-5. Use the alarm description tables, alarm statistics, and the database of historical alarms if necessary to help interpret the data and troubleshoot network problems. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Use the following procedures to troubleshoot network problems using alarms. Procedure 3-1 To view and sort alarms in the dynamic alarm list Monitor the dynamic alarm list in the 5620 SAM alarm window and attempt to address alarms in the order that they are generated.
Page 40
Table 3-26 for SNMP domain Table 3-25 for site security Table 3-22 for security domain Table 3-27 for software domain Table 3-30 for templates Table 3-31 for virtual scheduler domain Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Click on the Faults tab button, then click on the Related Problems tab button to display the alarms related to the affected object, as shown in Figure 3-2. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Alarms are generated by managed objects. Objects with alarms are called affected objects. Double-click on the selected alarm in the dynamic alarm list. The Alarm Info form opens as shown in the example in Figure 3-3. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 43
Correct the specified problem and all correlated alarms should clear. See section 3.4 for a sample configuration problem. If alarms still exist, perform Procedure 3-5. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Find the object type that is lowest in the network object hierarchy. See the object hierarchy in Table 3-1. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Equipment Down alarm • underlying port state problem that causes a number of related alarms at the LSP level • configuration problem that causes a Frame Size Problem alarm Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
In this example, the alarm to investigate is one of the first two detected Physical Port alarms: Port 1/1/2 on Site ID 10.1.200.52. Select the alarms related to this affected object and acknowledge the alarms. 3-10 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
This procedure describes how to clear the 22 alarms from the sample problem in this section. The troubleshooting process determined that two physical ports in LAG 2 at Site 10.1.200.52. are operationally down. Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-11 May 2005...
In this example, the lowest-level object type in the alarm list is Lsp Path in the Path/Routing Management domain. There are two Lsp Path Down alarms. One was generated later than the other. 3-12 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 49
Click on the Related Objects tab button on the Alarm Info form to display the managed objects related to the object in alarm, shown in Figure 3-9. Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-13 May 2005...
Page 50
Click on the Port tab button to view the underlying port state of the LAG member, as shown in Figure 3-10. The LAG Member 1/1/2 properties form shows the Underlying Port State: Shut Down. 3-14 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 51
In the navigation tree Equipment view, choose port 1/1/2 under the Shelf object that supports LAG 1 (Site 10.1.200.53), and choose Properties from the contextual menu. The properties form opens, as shown in Figure 3-11. Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-15 May 2005...
16 automatically clear. The remaining two alarms are Session alarms. Choose one of the remaining alarms and choose Show Affected Object from the contextual menu. The affected object properties form opens. 3-16 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Alarm Info form. Review the information about the alarm. • Affected object is circuit. • Alarm type is configuration alarm. • Probable cause is frame size problem. • Domain is Service Tunnel Management. Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-17 May 2005 95-5887-01-00-B...
Procedure 3-11 To clear a Frame Size Problem (MTU Mismatch) alarm This procedure describes how to clear the circuit Frame Size Problem alarm described in this section. Choose Service Management→Browse Services from the 5620 SAM main menu. 3-18 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005...
Page 55
Site form will not be applied to the service unless you click on the OK or Apply button in the Service form. Figure 3-15 Warning to apply changes to all objects Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-19 May 2005...
Alarm description tables Alarms are grouped by domain. Tables 3-2 to 3-31 describe the network object alarms that are raised on the 5620 SAM and are listed in domain alphabetical order. Table 3-2 Bundle domain alarm Alarm name, ID, type, and default...
Page 57
ID: 146 Object type: Card image on the router. Type: Firmware Alarm Domain: equipment Probable cause: Boot Rom Version Mismatch, FPGA Version Mismatch (1 of 2) Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-21 May 2005 95-5887-01-00-B...
Page 58
5620 SAM client GUI, as described in Procedure 8-4. Name: Threshold Crossing Alarm Severity: warning — ID: 14 Object type: Generic Object Type: Threshold Crossed Domain: generic Probable cause: Threshold Crossed 3-22 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 59
There is a configuration problem on Layer 3 forwarding service site. ID: 142 Object type: Service Site Type: Configuration Alarm Domain: l3fwd Probable cause: Route Distinguisher Not Configured Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-23 May 2005 95-5887-01-00-B...
Page 60
Name: Ldp Targeted Peer Down Severity: critical This is an LDP configuration component. ID: 23 Object type: Targeted Peer Type: Protocol Alarm Domain: ldp Probable cause: Protocol Down 3-24 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 61
Name: Inband Management Connection Severity: critical — Down Object type: Node Discovery Control ID: 139 Domain: netw Type: Communication Alarm Probable cause: Management Connection Down (1 of 4) Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-25 May 2005 95-5887-01-00-B...
Page 62
Name: Persistent Index Parameter Severity: major — Misconfigured Object type: Network Element ID: 173 Domain: netw Type: Configuration Alarm Probable cause: Persistent Index Configuration Mismatch (2 of 4) 3-26 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 63
ID: 33 Object type: Network Element pointing to 5620 SAM. Type: Configuration Alarm Domain: netw Probable cause: Trap Destination Misconfigured (3 of 4) Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-27 May 2005 95-5887-01-00-B...
Page 72
Domain: svc Probable cause: Topology Misconfigured Name: Type Mismatch Severity: critical — ID: 96 Object type: Service Type: Configuration Alarm Domain: svc Probable cause: Service Site Type Misconfigured 3-36 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 73
Additional information probable cause Name: Mediation Authentication Failure Severity: critical — ID: 75 Object type: Poller Manager Type: Communications Alarm Domain: snmp Probable cause: No Mediation Policy Found Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-37 May 2005 95-5887-01-00-B...
Page 74
Type: Equipment Alarm Domain: sw Probable cause: boot Environment Sync Failed Name: Bootable Config Backup Failed Severity: major 5620 SAM failed to back up the 7x50 node configuration files. ID: 103 Object type: Backup Restore Manager Type: Configuration Alarm Domain: sw...
Page 75
ID: 83 Object type: Sonet Port Specifics shut down. Type: Communications Alarm Domain: sonetequipment Probable cause: Loss Of Clock (1 of 3) Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-39 May 2005 95-5887-01-00-B...
Page 76
When configured, ID: 91 Object type: Sonet Port Specifics SLOS alarms are raised and Type: Communications Alarm Domain: sonetequipment cleared. Probable cause: section Loss Of Signal (2 of 3) 3-40 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 77
S1 byte. When configured, SS1F ID: 92 Object type: Sonet Port Specifics alarms are raised and cleared. Type: Communications Alarm Domain: sonetequipment Probable cause: Tx Section Synchronization Error (3 of 3) Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-41 May 2005 95-5887-01-00-B...
Page 78
Name: DS3E3 Loss Of Signal Severity: major — ID: 116 Object type: DS3E3 Channel Specifics Type: Communications Alarm Domain: tdmequipment Probable cause: loss Of Signal (1 of 2) 3-42 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 79
Type: Configuration Alarm Domain: vs scheduler with scheduler policy that Probable cause: undefined Scheduler is specified in the Scheduler tab. Reference Alcatel 5620 Service Aware Manager, Release 2.1 R2 3-43 May 2005 95-5887-01-00-B...
Page 80
3 — Troubleshooting network alarms 3-44 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
4 — Troubleshooting services 4.1 5620 SAM troubleshooting support for services 4.2 Workflow to troubleshoot a service problem with no associated alarms 4.3 Service troubleshooting menus 4.4 Service troubleshooting procedures Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
MAC Ping The procedures in this chapter use some of the OAM diagnostic tools in the workflow to troubleshoot a service. See the 5620 SAM User Guide for descriptive information and how to enable and access the OAM diagnostics. You must run the OAM diagnostic tools in both directions to Note —...
Verify the connectivity of the customer equipment using the entries in the FIB. See Procedure 4-3. Verify that the 5620 SAM service configuration aligns with the customer requirements. For example, ensure that 5620 SAM configuration uses the correct service type and SAP configuration, and that the circuit and site are included in the service.
Page 84
Verify the QoS configuration. You have completed the workflow for troubleshooting services. Contact your Alcatel technical support representative if the problem persists. See section 1.4 for more information. If the MAC Ping and MAC Trace diagnostics did not return the expected results...
Procedure 4-1 To identify if the service is part of an H-VPLS configuration Choose Service Management→Browse Services from the 5620 SAM main menu. Configure the list filter parameters and click on the Search button. A list of services appears at the bottom of the Browse Services form.
6 in section 4.2. If the operational state is Down and the administrative state is Up for one or more service components, the 5620 SAM generates an alarm. You must investigate the root problem on the underlying object. See chapter 3 for more information.
If there are no FIB entries, there is a configuration problem with the customer equipment or the connection from the equipment to the service tunnel. Confirm that the 5620 SAM service configuration aligns with the customer requirements. Confirm that there are no problems with the customer equipment and associated configuration.
Procedure 4-4 To verify connectivity for all egress points in a service using MAC Ping and MAC Trace Enable the OAM diagnostics for the service. See the 5620 SAM User Guide. Open the MAC Ping configuration form and clear the results from the previous diagnostic session, if any.
Page 89
ID 10.1.200.51/32 to site ID 10.1.200.52/32 using the network in Figure 4-1. Figure 4-5 MAC Ping configuration form for a node-to-node broadcast Figure 4-6 shows the response associated with the diagnostic configuration in Figure 4-5. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 90
MAC address and cannot flood the request to other routers. malformedEchoRequest (8) The received echo request is malformed. tlvNotUnderstood (9) One or more TLVs were not understood. 4-10 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 91
MAC Trace from site ID 10.1.200.51/32 to site IDs 10.1.200.52/32 and 10.1.200.53/32 using the network in Figure 4-1. Figure 4-7 MAC Trace configuration form Figure 4-8 shows the response associated with the diagnostic configuration in Figure 4-7. Alcatel 5620 Service Aware Manager, Release 2.1 R2 4-11 May 2005 95-5887-01-00-B...
Procedure 4-5 To measure frame transmission size on a service using MTU Ping Enable the OAM diagnostics for the service. See the 5620 SAM User Guide. Record the maximum frame transmission size for the service. Make sure that the OAM diagnostics are enabled on both circuits for the service tunnel.
Page 93
MTU Ping from site ID 10.1.200.52/32 to site ID 10.1.200.53/32 using the network in Figure 4-1. Figure 4-9 MTU Ping configuration form Figure 4-10 shows the response associated with the diagnostic configuration in Figure 4-9. Alcatel 5620 Service Aware Manager, Release 2.1 R2 4-13 May 2005 95-5887-01-00-B...
See section 1.4 for more information. Procedure 4-6 To verify the end-to-end connectivity of a service using Circuit Ping Enable the OAM diagnostics for the service. See the 5620 SAM User Guide. 4-14 Alcatel 5620 Service Aware Manager, Release 2.1 R2...
Page 95
Figure 4-11. Double-click on the entry in the Circuit Ping results form to view the diagnostic details. Figure 4-12 Circuit Ping results Review the diagnostic results and assess whether the configuration meets the network requirements. Table 4-3 lists the displayed messages. Alcatel 5620 Service Aware Manager, Release 2.1 R2 4-15 May 2005 95-5887-01-00-B...
Page 96
If the problem no longer exists on the service, you have completed the troubleshooting workflow for services. Disable the OAM diagnostics to conserve system resources. See the 5620 SAM User Guide. • If the service problem persists, another type of problem may exist on your service.
Procedure 4-7 To verify the end-to-end connectivity of a service tunnel using Tunnel Ping Enable the OAM diagnostics for the service. See the 5620 SAM User Guide. Open the Tunnel Ping configuration form and clear the results from the previous diagnostic session, if any.
Page 98
The request was not sent because the originating SDP administrative state is Down. Orig-SDP Oper-Down The request was not sent because the originating SDP operational state is Down. (1 of 2) 4-18 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Procedure 4-8 To verify end-to-end connectivity of an MPLS LSP using LSP Ping Enable the OAM diagnostics for the service. See the 5620 SAM User Guide. Open the LSP Ping configuration form and clear the results from the previous diagnostic session, if any.
Page 100
The far-end egress point exists and is operating correctly. No action required. fecNoMap (2) The replying router has no mapping for the FEC. notDownstream (3) The replying router is not a downstream router. (1 of 2) 4-20 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Procedure 4-9 To review the route for an MPLS LSP using LSP Trace Enable the OAM diagnostics for the service. See the 5620 SAM User Guide. Open the LSP Trace configuration form and clear the results from the previous diagnostic session, if any.
Page 102
If the LSP Trace returned the expected results for the configuration of your network, you have completed the workflow for troubleshooting services. Contact your Alcatel technical support representative if the problem persists. See section 1.4 for more information. If the LSP Trace did not return the expected results for the configuration of your network, verify that the correct MPLS LSP is used for the service.
If the problem no longer exists on the service, you have completed the troubleshooting workflow for services. Disable the OAM diagnostics to conserve system resources. See the 5620 SAM User Guide. • If the service problem persists, another type of problem may exist on your service.
Page 104
4 — Troubleshooting services 4-24 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
5 — Troubleshooting alarms using topology maps Network topology map overview Several network topology maps are available on the 5620 SAM. The maps display network objects. You can open contextual menus and submenus to open forms with additional information. For more information about topology maps, see the 5620 SAM User Guide.
Color and letter that indicate the most severe alarm on the device. Upper right corner Symbol that indicates connectivity to the device. This symbol corresponds to the icon color. Table 5-4 lists icon symbols and colors for 5620 SAM alarms. Table 5-4 Map alarm status indicators Map icon Alarm...
Use this procedure to diagnose a network element that has an alarm using one of the maps. Select the object with the alarm that you want to diagnose. Right-click to view the contextual menu. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 109
Click on the Edit button. The property form for the selected object opens. Click on the Faults tab button. The Faults tab form opens. View alarm status and diagnose the problem, as described in chapter 3. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 110
5 — Troubleshooting alarms using topology maps Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Bandwidth requirements vary depending on the type of management links set up, and the number of devices in the managed networks. For information about network planning expertise, contact your Alcatel technical support representative. Table 6-1 lists the minimum bandwidth requirements in the network management domain.
Review the output. The following shows sample output. # ping -s name_of_machine PING name_of_machine: 56 data bytes 64 bytes from name_of_machine (138.120.106.169): icmp_seq=0, Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
• routing using netstat and ARP The 5620 SAM uses numerous TCP and UDP ports for communication between various services. Some of these ports, such as the SNMP trap port, are configured during installation. Other ports are configured automatically by the software. Check that these ports are open or protected by a firewall, depending on system architecture needs.
Page 117
Run the netstat -r and arp -a commands to display active TCP connections, Ethernet statistics, the IP routing table, and the ports on which the PC or workstation is listening. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 118
6 — Troubleshooting network management LAN issues Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
7 — Troubleshooting Solaris and Windows platforms 7.1 Troubleshooting Solaris platforms 7.2 Troubleshooting Windows platforms 7-10 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
CPU usage data. You can also perform other procedures: • If you are you performing a large listing operation using the 5620 SAM client GUI or OSS, check the LAN throughput using the netstat command, as described in Procedure 8-1.
Page 121
WCPU COMMAND The top 5620 SAM process listed under the CPU column should be the Java process. However, the Java process should not be consuming too much CPU. Some Oracle processes could also take CPU time, depending on the database load.
Page 122
If processes are competing for CPU resources, you can isolate the information about a single process using the ps command. Check the state of CPUs by typing: /usr/ucb/ps -aux ↵ A list of processes appears. Review the ps output. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
To collect data to determine whether there is a disk bottleneck, type: iostat -x time ↵ where time is the time, in seconds, over which you want to collect data. Alcatel recommends that you start with 2 s. To stop the iostat command, press CTRL-C.
This scenario may continue until the system is no longer running any processes and is spending almost all of its time copying code and data in and out of main memory. Open a command or shell tool. Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 125
To collect data, type: vmstat s ↵ where s is the time, in seconds, over which you want to collect data. Alcatel recommends that you start with 2 s. Review the vmstat output. The following is a sample of vmstat data. See Table 7-3 for a description of the vmstat report.
Determine where there is enough disk space to create a swap file. Change directories by typing: cd /swapdirectory ↵ where swapdirectory is the name of the directory where you are going to create a new swap file Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 127
-l ↵ swap -s ↵ Several lines are displayed. The format of the last line is: total: 52108k bytes allocated + 24944k reserved = 77052k used, 93992k available Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
7 — Troubleshooting Solaris and Windows platforms Troubleshooting Windows platforms Many of the commands in section 7.1 and throughout the rest of the 5620 SAM Troubleshooting Guide can also be performed on a Windows platform PC. In all cases, the commands are run from the DOS command line. As well, you can check PC performance and running process details using the Task Manager.
To collect data to determine whether there is network bottleneck, type: netstat -i s ↵ where s is the time, in seconds, over which you want to collect data. Alcatel recommends that you start with 50 s; this time interval may require adjusting to meet your specific requirements.
The client cannot print The printer was not added to the list of Add the printer to the list of printers by using available printers the admintool Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Procedure 8-3 Problem: I discovered a new router, but cannot place it in a managed state Possible causes are: • an incorrect 5620 SAM server license key was entered or the license key is corrupt • the 5620 SAM server license key is not for the correct hostid •...
9 for more information. • Failed deployment of the configuration request. For the 5620 SAM client, perform the following: Choose Mediation→Deployment and Site Backup/Upgrade from the 5620 SAM main menu. The Deployment and Site Backup/Upgrade form with the Deployers tab button displayed appears.
Attribute: alarmClassTag Value: generic.DeploymentFailure The alarm also contains additional information, including the object affected by the alarm and the severity of the alarm. See the Alcatel 5620 SAM-O OSS Interface Developer Guide for more information. Find the following text in the alarm: Attribute: requestID=requestID The parameter specifies the request id sent with the original request.
The server PC or workstation, or the server process on the PC or workstation, is not responding correctly, or is not running. To check that the 5620 SAM client points to the correct IP address and port of the server, open the nms-client.xml file using a text editor. The default file location is install directory/nms/config.
• the UNIX user of the 5620 SAM client has the same group permissions as the UNIX user who installed the 5620 SAM client. To check that the 5620 SAM client and server versions are the same: Check the version of the 5620 SAM client by choosing Help→About from the...
When a client executes a request to the server to provide a large amount of data, such as requests for a large number of logged statistics, the 5620 SAM server may be unable to process the request if the query limit size is exceeded. A warning message is presented to the user on the client GUI, or an OSS client receives a SOAP invocation error indicating that the result set is too large.
5620 NM or 1354 BM Possible causes include incorrectly configured param.cfg parameters on the 5620 NM or 1354 BM to allow the forwarding of alarms to those platforms from the 5620 SAM. Open a command tool on the 5620 NM or 1354 BM.
You can perform the following: • from the 5620 SAM client GUI, admin users can disable the GUI inactivity check, if required. • from the 5620 SAM client xml configuration file, reconfigure the client-server communication activity check to an appropriate time.
Procedure 8-12 Problem: I performed a search or list function, and it takes too long to complete You can perform simple or complex searches using the Find menu on the 5620 SAM main menu to query the database for information about services, subscribers, and other stored data.
Procedure 8-13 Problem: I cannot select certain menu options or I cannot save configurations The 5620 SAM allows the administrator to restrict access to parts of the GUI, or restrict the ability of a user to configure objects or save configurations. Check with your administrator to determine your privileges.
Choose to ignore timestamps. Click on the Resync button. Stop and start the client GUI, as described in the 5620 SAM User Guide. Run the nmsserver status script, as described in Procedure 9-4. Check that sufficient free memory is available.
Page 143
5620 SAM User Guide • the server is properly configured to run when SSL is enabled, as described in Procedure 9-12 Alcatel 5620 Service Aware Manager, Release 2.1 R2 8-15 May 2005 95-5887-01-00-B...
Page 144
8 — Troubleshooting 5620 SAM client GUI 8-16 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
9 — Troubleshooting 5620 SAM server issues 9.1 Troubleshooting 5620 SAM server issues Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
(MDAs), you can dynamically update the license key without shutting down the server. Check the dynamic alarm list on the 5620 SAM client GUI or the JMS real-time alarm feed from the 5620 SAM OSS client application for critical alarms related to exceeding the license limits.
Page 147
Check the amount of swap space available. Insufficient swap space can prevent the 5620 SAM server from launching. See Procedure 7-4 for more information. Specific ports need to be available for the 5620 SAM server. See Procedure 6-4 for more information about specific port values.
When an intermediate network device, such as a router, receives the management traffic to and from the 5620 SAM and the managed network, it must be able to process packets of up to 9216 bytes. If this exceeds the MTU for the intermediate device, or if the device cannot perform packet fragmentation, then large packets may be dropped and resynchronization may fail.
9 — Troubleshooting 5620 SAM server issues On Solaris installations, If you cannot ping the 5620 SAM server, make sure that the host name of the server is in the /etc/hosts file. Change to the /etc directory by typing: cd /etc ↵...
Launch the nmsserver.bash or nmsserver.bat executable with the following flag: nmsserver.* executable flag ↵ where nmsserver.* is either nmsserver.bash or nmsserver.bat executable flag is one of the options in Table 9-1 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 151
Table 9-1 nmsserver.* flag options Flag option Description daemon Starts the 5620 SAM server in non-interactive mode. This is the recommended usage. Not using a flag also starts the server in interactive mode. start Starts the 5620 SAM server in interactive mode.
Page 152
• the MaxMemory and TotalMemory values match or are close in value, otherwise other applications may be affecting available memory for use by the 5620 SAM applications •...
5620 SAM server, or no SNMP traps are arriving When you install the 5620 SAM server, you specify a port where SNMP traps arrive. In addition, two sets of configurations must be completed for SNMP trap notifications to work: •...
Page 154
• an SNMP trap group • that the SNMP trap group is associated with the IP address of the 5620 SAM server Go to the install directory/nms/config directory or folder to check that SNMP values are correctly set to enable traps to reach the 5620 SAM server.
Procedure 9-8 Problem: Unable to receive alarms on the 5620 NM from the 5620 SAM Check that the 5620 NM AS tool is properly configured to receive alarms from the 5620 SAM. Alcatel 5620 Service Aware Manager, Release 2.1 R2...
Ensure that the integration software is properly configured, as described in the 5620 SAM Installation and Upgrade Guide. Configure the param.cfg file on the 5620 NM to ensure that alarms are forwarded from the 5620 SAM to the 5620 NM AS tool: Open a command tool on the 5620 NM.
• FTP must be enabled on the managed device in order for the 5620 SAM to retrieve statistics, and the user logged into the 5620 SAM must have FTP permissions on the managed device, as specified in the 5620 SAM User Guide Alcatel recommends that statistics collection planning includes the following considerations, to prevent the loss of statistics interval data.
Go to the install directory/nms/config directory or folder and locate the nms-server.xml file to check that redundant database configuration values are set correctly to allow the 5620 SAM server to communicate with the primary (active) database and, in the case of a database failure, the standby database.
.bash file with the correct java virtual machine argument to include the path to the certificate keystore, as described in the 5620 SAM User Guide • the server is restarted after the nmsserver.bat file is updated Use the following java virtual machine statement in the appropriate *.bat or *.bash file.
Page 160
9 — Troubleshooting 5620 SAM server issues If the keystore file is under the jboss directory, modify the *.bat or *.bash file to modify the JVM_HIGH_OPTIONS, as described in the 5620 SAM User Guide. The following shows an example for the *.bat file.
Verify that the database platform is adequately sized. The minimum platform requirements are available in the appropriate release notice or the 5620 SAM Planning Guide, available from your Alcatel support representative. Check the partition or root database backup directory to ensure that: •...
RMAN>exit; ↵ Perform a database backup using the 5620 SAM client GUI, as described in the 5620 SAM User Guide, or using the DBconfig installer, as described in the 5620 SAM Installation and Upgrade Guide. Store the database backup in a secure location.
Alcatel warranty and support agreements. Contact your Alcatel technical support representative to help you troubleshoot your database. As the oracle user, launch the 5620 SAM database configuration tool from the appropriate directory on the product DVD, as described in the 5620 SAM Installation and Upgrade Guide.
Specify whether to create a copy of the backed up database. When the backup database is restored, Oracle modifies the backup and it cannot be reused. Specify any additional parameters, as listed in the 5620 SAM Installation and Upgrade Guide.
Performing such modifications can void your Alcatel warranty and support agreements. Contact your Alcatel technical support representative to help you troubleshoot your database. Database backupsets expire based on a retention period. The default retention period is seven days.
• Ensure that there are identical operating system and versions of the 5620 SAM software installed on the active and standby database workstations. Ensure there are no LAN communication problems between the active and standby database platforms.
Page 168
For the Listener, type: ps -ef|grep tnslsnr ↵ One entry should be shown for the Listener. For the database, type: ps -ef|grep ora ↵ There should be multiple Oracle instances listed. 10-8 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
When an error condition is encountered that the 5620 SAM client has not anticipated, a Problems Encountered window is displayed. See section 12.1 for more information.
11 — 5620 SAM client GUI warning message output Figure 11-2 Password missing warning dialog box The warning message indicates the information that is required. In this case, click on the OK button to close the dialog box, and configure the New Authentication Password and Confirm New Auth Password parameters.
11 — 5620 SAM client GUI warning message output Figure 11-4 Committing changes to resources warning dialog box When you click on the Yes button for the last confirmation the changes to the parent or sub-forms are committed. Service disruption warning A service disruption dialog box is displayed when you perform an action that may be service-affecting.
11 — 5620 SAM client GUI warning message output Duplicate configuration form conflicts There are multiple ways to access a configuration form for the same object. For example, you can view the configuration form for a service by choosing Service Management→Browse Services, or you can access the service by clicking on the...
Page 174
11 — 5620 SAM client GUI warning message output Correct the problem based on the information provided. For the example in Figure 11-8, configure the authentication order parameters. If you cannot correct the problem and continue to get the same warning message: Check the documentation to ensure that you are following the steps correctly.
12 — Troubleshooting with Problems Encountered forms 12.1 Problems Encountered form overview 12-2 12.2 Using Problems Encountered forms 12-3 Alcatel 5620 Service Aware Manager, Release 2.1 R2 12-1 May 2005 95-5887-01-00-B...
For additional information, click on the Edit button. The information may not be enough for you to correct the problem. The information can be used by your technical support representative to help resolve the problem. 12-2 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Procedure 12-2 To collect problem information for support The following procedure describes what to do before you contact your contact technical support representative when you cannot resolve a problem on the Problems Encountered form. Alcatel 5620 Service Aware Manager, Release 2.1 R2 12-3 May 2005 95-5887-01-00-B...
Page 178
Some long-running applications can generate multiple log files. Most log files are stored in the install_directory/version/nms/log directory or folder. Store the files in a secure location. Send the information collected to your technical support representative. 12-4 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
13 — Troubleshooting with the client activity log 13.1 Activity Log Manager form overview 13-2 13.2 Using Activity Log Manager forms 13-4 Alcatel 5620 Service Aware Manager, Release 2.1 R2 13-1 May 2005 95-5887-01-00-B...
Activity Log Manager form overview The Activity Log Manager allows users with administrative privileges to view user activity for 5620 SAM GUI and OSS clients. Figure 13-1 shows the Activity Log Manager form. Figure 13-1 Activity Log Manager form details Table 13-1 describes the types of logs available in the Activity Log Manager form.
Page 181
5620 SAM-O interface to export log data in an XML format. (Filtered lists of log entries can also be retrieved through the 5620 SAM-O interface.) You can use the XML log data as an archive mechanism or statistical analysis method. See the Alcatel 5620 SAM-O OSS Interface Developer Guide for more information.
Violation of user permissions through the XML OSS interface Note The 5620 SAM also raises an alarm for log entries that are related to security. 13.2 Using Activity Log Manager forms The following procedures describe how to use the Activity Log Manager form to correlate user requests and deployment activity.
Use the filter in the User Request Log (userlog) to list the requests for the request ID that you obtained in step 5. The 5620 SAM displays the filter results in the Userlog display area. The Operation Source column identifies the user associated with the failed deployment.
13 — Troubleshooting with the client activity log Procedure 13-4 To retrieve historical user logs Choose Find→Browse Log Records from the 5620 SAM main menu. The Browse Log Records form appears. Set the Log Class parameter to the type of user log you want to view, as listed in Table 13-1.
Page 185
Glossary Numerics 5620 SAM 5620 Service Aware Manager The 5620 SAM is the network manager portfolio of modules for the 7750 SR and 7450 ESS. 5620 SAM client The 5620 SAM client provides a GUI to configure IP network elements.
Page 186
FTP generally runs over TCP or UDP. graphical user interface A GUI is a computer user interface that incorporates graphics to make software easier to use. IETF Internet engineering task force GL-2 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 187
The MTU can change over a network. navigation tree The navigation tree displays a view of all managed equipment, services, and protocols, and allows you to navigate through these components. network element Alcatel 5620 Service Aware Manager, Release 2.1 R2 GL-3 May 2005 95-5887-01-00-B...
Page 188
A network topology is the layout of a network, which can include the way in which elements in a network, such as nodes, are connected and how they communicate. networkstation A UNIX platform where the 5620 SAM software runs. NPDU network protocol data unit operational support system A network management system supporting a specific management function, such as alarm surveillance and provisioning, in a service provider network.
Page 189
The SSH protocol is used to protect communications between two hosts by encrypting a Telnet or FTP connection between the 5620 SAM and some nodes. 5620 SAM uses SSH version 1.5. Both ends of the client/server connection are authenticated, and passwords are protected by being encrypted.
Page 190
Windows commonly allow a user to input data and initiate functions but some windows simply display information. workflow The 5620 SAM workflow is a defined series of tasks that describe how to install, configure, create, and manage services. X.733 ITU-T X.733...
Page 192
8-5 software (sw), 3-38 diagnostics, 4-2 SONET equipment, 3-39 discovery errors TDM equipment, 3-42 5620 SAM server, 9-11 template, 3-43 disk vs, 3-43 troubleshooting bottlenecks, 7-5 IN-2 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 193
5620 SAM client, 8-5 troubleshooting, 1-4 memory, 7-8 troubleshooting bottlenecks, 7-6 memory bottlenecks, 7-6 H-VPLS identification, 4-5 message hierarchy of managed objects, 3-4 warning overview, 11-2 Alcatel 5620 Service Aware Manager, Release 2.1 R2 IN-3 May 2005 95-5887-01-00-B...
Page 194
9-2 5620 SAM database, 10-7 service tunnels OSS client verifying connectivity, 4-17 troubleshooting, 8-2 services FIB configuration, 4-7 H-VPLS identification, 4-5 menus, 4-5 status of components, 4-6 IN-4 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Page 195
9-5 server communications, 9-4 top command, 7-2 server performance, 9-5 traceroute, 6-4 server shutdowns, 9-11 tracert, 6-4 server status, 9-6 traps, 9-9 server to database communications, 9-12 Alcatel 5620 Service Aware Manager, Release 2.1 R2 IN-5 May 2005 95-5887-01-00-B...
Page 196
8-8 version compatability, 8-8 vmstat command, 7-6 warning message additional information required, 11-2 commitment of changes from a form and its sub-forms, 11-3 IN-6 Alcatel 5620 Service Aware Manager, Release 2.1 R2 May 2005 95-5887-01-00-B...
Customer documentation http://www.alcatel.com/osds/ Product manuals and documentation updates are available through the Alcatel Support Documentation and Software Download service at Alcatel.com. If you are a new user and require access to this service, please contact your Alcatel sales representative. Technical support http://www.alcatel.com/support/...
Need help?
Do you have a question about the 5620 and is the answer not in the manual?
Questions and answers