Location Of Tracelogs; Tracelog Naming Convention; Rotation And Throttling Policy - Cisco Catalyst 3650 series Command Reference Manual

Information About Tracing

Location of Tracelogs

Each process uses btrace infrastructure to log its trace messages. When a process is active, the corresponding
in-memory tracelog is found in the directory /tmp/<FRU>/trace/, where <FRU> refers to the location
where the process is running (rp, fp, or cc).
When a tracelog file has reached the maximum file size limit allowed for the process, or if the process ends,
it gets rotated into the following directory:
• /crashinfo/tracelogs, if the crashinfo: partition is available on the switch
• /harddisk/tracelogs, if the crashinfo: partition is not available on the switch
The tracelog files are compressed before being stored in the directory.

Tracelog Naming Convention

All the tracelogs that are created using btrace have the following naming convention:
<process_name>_<FRU><SLOT>-<BAY>.<pid>_<counter>.<creation_timestamp>.bin
Here, counter is a free-running 64-bit counter that gets incremented for each new file created for the process.
For example, wcm_R0-0.1362_0.20151006171744.bin. When compressed, the files will have the gz extension
appended to their names
Tracelog size limits and rotation policy
The maximum size limit for a tracelog file is 1MB for each process, and the maximum number of tracelog
files that are maintained for a process is 25.

Rotation and Throttling Policy

Initially, all the tracelog files are moved from the initial /tmp/<FRU>/trace directory to the
/tmp/<FRU>/trace/stage staging directory. The btrace_rotate script then moves these tracelogs
from the staging directory to the /crashinfo/tracelogs directory. When the number of files stored in
the /crashinfo/tracelogs directory per process reaches the maximum limit, the oldest files for the
process are deleted, while the newer files are maintained. This is repeated at every 60 minutes under worst-case
situations.
There are two other sets of files that are purged from the /crashinfo/tracelogs directory:
• Files that do not have the standard naming convention (other than a few exceptions such as
fed_python.log)
• Files older than two weeks
The throttling policy has been introduced so that a process with errors does not affect the functioning of the
switch. Whenever a process starts logging at a very high rate, for example, if there are more than 16 files in
a 4-second interval for the process in the staging directory, the process is throttled. The files do not rotate for
the process from /tmp/<FRU>/trace into /tmp/<FRU>/trace/stage, however the files are deleted
when they reach the maximum size. Throttling is re-enabled, when the count goes below 8.
Command Reference, Cisco IOS XE Everest 16.5.1a (Catalyst 3650 Switches)
984