Send in your Unix questions today! |
See additional Unix tips and tricks
Several weeks ago, I encouraged readers to automate the extraction of warning messages from their /var/adm/messages (or /var/log/messages) files and provided a script to do just that. In this article, we will look at a re-implementation of that script.
The new version of this script was written by Jared Still, Certifiable Oracle DBA and Part Time Perl Evangelist and author of "Perl for Oracle DBAs". Jared's rewrite of the look4warnings script illustrates how elegantly Perl can be used to extract and summarize the contents of log files.
The first commands in the script rewrite are, more or less, the same as in the original version. We assign our messages file to a variable ($msgs) and attempt to open the file. If we run into problems, we exit, displaying a message like this generated by the die command:
Cannot open /var/log/messages - No such file or directory
|
The script, then, begins with a comment and the lines to set up our input file.
#!/usr/bin/perl -w
# look through messages file for warnings, show summaries
$msgs="/var/log/messages";
open (MSGS,"<$msgs") || die "Cannot open $msgs - $!\n";
|
The extraction of all lines containing the word "warning", on the other hand, is accomplished far more simply and elegantly than in the original version. Notice that we accomplished this using a single command.
# get all warnings
my @warnings=grep(/warning/i,<MSGS>);
|
At this point, every line in the messages file that contains the word "warning" in any combination or uppercase and lowercase letters has been added to the @warnings array. This includes duplicate warnings with the same or different timestamps. If a particular warning has arrived once a day for a week, for example, that warning will be saved in the array seven times.
The following command then strips the time stamps off of each line with the map command. This particular map command is preserving in each element of the array the text beginning with the word "warning" (or "WARNING", "Warning" and so on) as indicated by the "i" following the pattern we are matching.
Specifically, warning matches "warning", \s* matches some amount of white space, (.*) matches anything (i.e., the rest of the line) and "i" says to ignore case.
Once this command is run, we have the same number of elements in the @warnings array, but each has been shortened, the time stamp having been stripped.
# get string without timestamp
@warnings = map( /warning:\s*(.*)/i, @warnings);
|
Perl's map function is perfect for simplifying potentially repetitive operations, such as capitalizing strings of text or selecting some portion of each line.
If no warnings were found in the messages file, we will issue a message to this effect and exit. We exit with a return code of 0 since no error is indicated by a messages file that doesn't contains warnings.
unless (@warnings) {
print "No warnings found in $msgs\n";
exit 0;
}
|
Next, we do the trickier part. We set %warncount up as an empty hash and then use it to accumulate a count of each warning message. $warncount{$_}++ is a count of the current warning message, incremented each time a particular warning is encountered as we move through the array.
# create hash with count
%warncount=();
%warnings = map{$_,$warncount{$_}++} @warnings;
Finally, we display the list of warning messages along with the number of times each warning appears in the messages file:
foreach my $warning ( sort keys %warnings ) {
print "warning: $warning: ", $warnings{$warning}+1,"\n";
}
If you want to also display a count of the unique warnings found in the file, you can add this to the bottom of the script:
print "\nunique warning messages: $#warnings\n";
|
For your cutting and pasting pleasure, here's the script in non-interrupted format:
#!/usr/bin/perl -w
# look through messages file for warnings, show summaries
$msgs="/var/log/messages";
open (MSGS,"<$msgs") || die "Cannot open $msgs - $!\n";
# get all warnings
my @warnings=grep(/warning/i,<MSGS>);
# get string without timestamp
@warnings = map( /warning:\s*(.*)/i, @warnings);
unless (@warnings) {
print "No warnings found in $msgs\n";
exit 0;
}
# create hash with count
%warncount=();
%warnings = map{$_,$warncount{$_}++} @warnings;
foreach my $warning ( sort keys %warnings ) {
print "warning: $warning: ", $warnings{$warning}+1,"\n";
}
|
For more on Jared's adeptness with Perl, I recommend his excellent text (with coauthor Andy Duncan) -- Perl for Oracle DBAs, O'Reilly, 2002 and his personal web site, http://www.jaredstill.com.
Sandra Henry-Stocker has been administering Unix systems
for more than 18 years. She describes herself as "USL"
(Unix as a second language) but remembers enough English
to write books and buy groceries. She
currently works for TeleCommunication Systems, a wireless
communications company, in Annapolis, Maryland, where no
one else necessarily shares any of her opinions. She lives
with her second family on a small farm on Maryland's
Eastern Shore. Send comments and suggestions to bugfarm@gmail.com.
|
|
