Automatic Restart - Cisco BTS 10200 Troubleshooting Manual

Chapter 16 Disaster Recovery Procedures

Automatic Restart

This section describes the new BTS 10200 Automatic Restart feature. The BTS 10200 Automatic Restart
feature is beneficial to customers because the BTS 10200 will attempt to automatically restart a platform
(EMS/FS/CA) to STANDBY that has become OOS-FAULTY and has shut down. Currently the BTS
10200 will not restart a platform in this situation, leaving the active platform running in a vulnerable
simplex mode until the standby platform is restored manually.
An automatic restart is not attempted if the restart of the system is likely to fail.
Note
Benefits of this feature:
•
•
•
The processing associated with the BTS 10200 Automatic Restart feature has two main phases. In the
first phase, the transition is made to OOS-FAULTY. During this phase, the forensic information for
offline analysis must be collected and stored. In the second phase, the attempts are made to bring the
platform from OOS-FAULTY to STANDBY.
Transition to OOS-FAULTY
When the platform is transitioning to OOS-FAULTY, the Automatic Restart feature requires the
additional processing described here.
There is no need to save the Alarm log or Event log because they are contained in the MySQL database.
This database is external to the platform and is not affected by platform shutdowns or bringups.
The following processes are completed during the transition to OOS-FAULTY:
•
•
•
•
•
OL-8723-19
Reduced outage risk. Automatic restart–when successful–brings the platform up to STANDBY in
minutes instead of potentially much longer, as support personnel work to restore the platform. This
reduces the risk of outages by reducing the amount of time the system is in simplex mode.
Automated forensic data collection. This feature automatically saves data useful for offline
debugging (trace logs, status files, cores, etc.). Currently this data is retrieved manually. Automating
the process of saving off useful information guarantees that this information is preserved.
Faster switchovers. When a process exceeds the maximum number of restarts a system-initiated
switchover is executed. By not transitioning to OOS-FAULTY, the standby side will avoid the taxing
database copy process.
Create /saved.debug Subdirectory—Create the /saved.debug subdirectory.
Archive the /data Directory—Copy and archive the /data directory and all of its contents to the
/saved.debug directory using a tar file with a timestamp in the name.
Unwritten Traces—Any unwritten, buffered trace memory is saved.
Save Trace Logs—The most recent trace log files are saved. The number of logs to save is a
configurable parameter. This is especially useful when the Automatic Restart feature is active
because a system side might only temporarily remain in OOS-FAULTY. If the system side comes
back to a working STANDBY state, it is less urgent that support personnel retrieve logs, etc. from
the node.
Disk Space for /saved.debug—The disk space consumed by /saved.debug is conserved by deleting
the /saved.debug directory and its contents each Automatic Restart cycle. The saved.debug
subdirectory is created in the platform's /bin directory, i.e., <path>/bin/saved.debug.
Cisco BTS 10200 Softswitch Troubleshooting Guide, Release 5.0.x
Automatic Restart
16-37