Recoverable System Restarts - Cisco 9134 - MDS Multilayer Fabric Switch Troubleshooting Manual

Chapter 2 Troubleshooting Installs, Upgrades, and Reboots
S e n d d o c u m e n t a t i o n c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m

Recoverable System Restarts

Every process restart generates a syslog message and a Call Home event. Even if the event does not affect
service, you should identify and resolve the condition immediately because future occurrences could
cause service interruption.
To respond to a recoverable system restart, follow these steps:
Enter the following command to check the syslog file to see which process restarted and why it restarted:
Step 1
switch# show log logfile | include error
For information about the meaning of each message, refer to the Cisco MDS 9000 Family System
Messages Reference.
The system output looks like the following example:
Sep 10 23:31:31 dot-6 % LOG_SYSMGR-3-SERVICE_TERMINATED: Service "sensor" (PID 704) has
finished with error code SYSMGR_EXITCODE_SY.
switch# show logging logfile | include fail
Jan 27 04:08:42 88 %LOG_DAEMON-3-SYSTEM_MSG: bind() fd 4, family 2, port 123, ad
dr 0.0.0.0, in_classd=0 flags=1 fails: Address already in use
Jan 27 04:08:42 88 %LOG_DAEMON-3-SYSTEM_MSG: bind() fd 4, family 2, port 123, ad
dr 127.0.0.1, in_classd=0 flags=0 fails: Address already in use
Jan 27 04:08:42 88 %LOG_DAEMON-3-SYSTEM_MSG: bind() fd 4, family 2, port 123, ad
dr 127.1.1.1, in_classd=0 flags=1 fails: Address already in use
Jan 27 04:08:42 88 %LOG_DAEMON-3-SYSTEM_MSG: bind() fd 4, family 2, port 123, ad
dr 172.22.93.88, in_classd=0 flags=1 fails: Address already in use
Jan 27 23:18:59 88 % LOG_PORT-5-IF_DOWN: Interface fc1/13 is down (Link failure
or not-connected)
Jan 27 23:18:59 88 % LOG_PORT-5-IF_DOWN: Interface fc1/14 is down (Link failure
or not-connected)
Jan 28 00:55:12 88 % LOG_PORT-5-IF_DOWN: Interface fc1/1 is down (Link failure o
r not-connected)
Jan 28 00:58:06 88 % LOG_ZONE-2-ZS_MERGE_FAILED: Zone merge failure, Isolating p
ort fc1/1 (VSAN 100)
Jan 28 00:58:44 88 % LOG_ZONE-2-ZS_MERGE_FAILED: Zone merge failure, Isolating p
ort fc1/1 (VSAN 100)
Jan 28 03:26:38 88 % LOG_ZONE-2-ZS_MERGE_FAILED: Zone merge failure, Isolating p
ort fc1/1 (VSAN 100)
Jan 29 19:01:34 88 % LOG_PORT-5-IF_DOWN: Interface fc1/1 is down (Link failure o
r not-connected)
switch#
Enter the following command to identify the processes that are running and the status of each process.
Step 2
switch# show processes
The following codes are used in the system output for the State (process state):
•
•
•
•
•
•
•
OL-9285-05
D = uninterruptible sleep (usually I/O)
R = runnable (on run queue)
S = sleeping
T = traced or stopped
Z = defunct ("zombie") process
NR = notrunning
ER = should be running but currently notrunning
Troubleshooting Cisco SAN-OS Software System Reboots
Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x
2-27