Restoring And Recovering Resources; Reinstalling The System; Patching The System; Reporting The Incident - Red Hat ENTERPRISE LINUX 4 - SECURITY GUIDE Manual

Chapter 10. Incident Response

10.5. Restoring and Recovering Resources

While an incident response is in progress, the CERT team should be investigating while working
toward data and system recovery. Unfortunately, it is the nature of the breach which dictates the
course of recovery. Having backups or offline, redundant systems during this time is invaluable.
To recover systems, the response team must bring any downed systems or applications back online,
such as authentication servers, database servers, and any other production resources.
Having production backup hardware ready for use is highly recommended, such as extra hard drives,
hot-spare servers, and the like. Ready-made systems should have all production software loaded and
ready for immediate use. Only the most recent and pertinent data needs to be imported. This ready-
made system should be kept isolated from the rest of the network. If a compromise occurs and the
backup system is a part of the network, then the purpose of having a backup system is defeated.
System recovery can be a tedious process. In many instances there are two courses of action from
which to choose. Administrators can perform a clean re-installation of the operating system on each
affected system followed by restoration of all applications and data. Alternatively, administrators can
patch the offending vulnerabilities and bring the affected system back into production.

10.5.1. Reinstalling the System

Performing a clean re-installation ensures that the affected system is cleansed of any trojans,
backdoors, or malicious processes. Re-installation also ensures that any data (if restored from a
trusted backup source) is cleared of any malicious modifications. The drawback to total system
recovery is the time involved in rebuilding systems from scratch. However, if there is a hot backup
system available for use where the only action to take is to dump the most recent data, system
downtime is greatly reduced.

10.5.2. Patching the System

Patching affected systems is a more dangerous course of action and should be undertaken with great
caution. The problem with patching a system instead of reinstalling is determining whether or not a
given system is cleansed of trojans, security holes, and corrupted data. Most rootkits (programs or
packages that a cracker uses to gain root access to a system), trojan system commands, and shell
environments are designed to hide malicious activities from cursory audits. If the patch approach is
taken, only trusted binaries should be used (for example, from a mounted, read-only CD-ROM).

10.6. Reporting the Incident

The last part of the incident response plan is reporting the incident. The security team should take
notes as the response is happening and report all issues to organizations such as local and federal
authorities or multi-vendor software vulnerability portals, such as the Common Vulnerabilities and
3
http://cve.mitre.org/
Exposures site (CVE) at . Depending on the type of legal counsel an enterprise
employs, a post-mortem analysis may be required. Even if it is not a functional requirement to a
compromise analysis, a post-mortem can prove invaluable in helping to learn how a cracker thinks and
how the systems are structured so that future compromises can be prevented.
3
http://cve.mitre.org
94