Chapter 9. Log Files

Contents:

If this weren't a book on system
administration, an entire chapter on log files would seem peculiar.
But system administrators have a very special relationship with log
files. Like Doctor Doolittle, who could talk to the animals, system
administrators are expected to be able to communicate with a large
menagerie of software and hardware. Much of this communication takes
place through log files, so we become log file linguists. Perl can be
a big help in this process.

It is impossible to touch on all of the different kinds of processing
and analysis you can do with logs. Entire books have been devoted to
just statistical analysis of this sort of data. However, this chapter
should give you some general approaches to the topic and Perl tools
to whet your appetite for more.

9.1. Text Logs

Logs come
in different flavors, so we need several approaches for dealing with
them. The most common type of log file is one composed entirely of
lines of text. Popular server packages like Apache (web), INN (Usenet
news), and Sendmail (email) spew log text in voluminous quantities.
Most logs on Unix machines look similar because they are created by a
centralized logging facility known as syslog. For
our purposes, we can treat files created by syslog
like any other text file.

Here's a simple Perl program to scan for the word
"error" in a text-based log file:

open(LOG,"logfile") or die "Unable to open logfile:$!\n";
while(<LOG>){
print if /\berror\b/i;
}
close(LOG);

Perl-savvy readers are probably itching to turn it into a one-liner.
For those folks: