So, let’s say you want to parse your own log files and extract some very useful information, but you (as every normal person should) don’t want to reinvent the wheel. Well, logwatch does everything you need except, of course, parsing the log files in your format or interpret your own messages. This is easy to fix, although documentation left me puzzling for about an hour or so. Hence this post.
Apparently this is not as straightforward as it should be for ‘just a bunch of (highly usable) perl scripts’, therefore my experience.
Install
If you don’t have logwatch installed (how come?) then do something like
# wget http://downloads.sourceforge.net/project/logwatch/logwatch-7.3.6.tar.gz?use_mirror=ignum
# tar xzf logwatch-7.3.6.tar.gz
# cd logwatch-7.3.6
# ./install_logwatch.sh
Normally you should already have one pre-installed.
Custom script
So, let’s say you want to parse your own log files and extract some very useful information, but you (as every normal person should) don’t want to reinvent the wheel. Well, logwatch does everything you need except, of course, parsing the log files in your format or interpret your own messages. This is easy to fix, although documentation left me puzzling for about an hour or so. Hence this post.
I will explain in the order how I have approached it. This means that I don’t create all kinds of configuration files FIRST like all regular HOWTO’s suggest. Not at all. First you create… a script to parse your log file! And test it of course… And THEN you think how to hook it up. And not the other way around.
Script file
You can use anything you want that can be executed and accept input from STDIN (so potentially I could have used Python), but I somehow decided to be consistent with the rest of the logwatch and carry on with perl. Well, there were days I thought it is a fine language (approximately until I’ve “The Pragmatic Programmer“), but now it took quite some shivers to get back in the perl mood…
Basically I’ve copied the existing script and started messing up with it, resulting in something like (no fancy stuff):
#!/bin/env perl
use strict;
# Will hold pairs (error_type_string, count)
my %ErrorTypeCount;
my $ThisLine;
# Go through the input.
while (defined($ThisLine = <STDIN>))
{
# Specify number of errors per exception type.
if ($ThisLine =~ /\[ERROR\].*?\/oi)
{
my $key = $1;
if (exists $ErrorTypeCount{$key})
{
$ErrorTypeCount{$key} += 1;
}
else
{
$ErrorTypeCount{$key} = 1;
}
}
}
# Print the total stats.
print "\nERRORS\n";
while (my ($type, $count) = each(%ErrorTypeCount))
{
print " * FOUND $count of $type\n";
}
print "TOTAL $TotalErrorsCount ERROR(s)\n";
The code above goes through the log file, takes any match of the error, which in my case looks like:
2010-01-01 [ERROR] Raising Exception: () my error description
It will count the errors per ‘class’ and then print a summary.
Inputs and outputs
Important to understand:
- All log files you will specify in the configuration files will be given as STDIN input to your script. So you just read from STDIN and do not bother which files these actually are.
- Any strings you print will become part of your logwatch mail sent to the corresponding account (root by default).
So in your script you focus only on what you really need to make – parsing your stuff, nothing else. No reinvention, only what is needed! Now save the script to e.g. my-service.pl. Think of a proper name, normally the name is given after the name of the service it will parse the log file from (e.g. ssh, or exim), but it would be strange to have ssh file in your source folder, isn’t it? So let give it some name with extension for the moment.
What’s next?
Of course, you test your script by feeding it with some sample log files like
# cat my.log | perl my-service.pl
Ok, now it all works perfectly (perhaps hours later, but that’s where your time has to go to, not in reading endless discussions on why logwatch itself does not do this or that…). Now you’re ready to deploy.
There will be three actions needed to deploy your script on CentOS integrated with the logwatch installation
1. Deploy the script itself
The script has to be copied next to the other scripts where logwatch expects them, I kept the naming convention (no extensions) here as well, so
# sudo cp my-service.pl /usr/share/logwatch/scripts/services/my-service
# chmod a+x /usr/share/logwatch/scripts/services/my-service
So you will end up with a new executable file under the /usr/share/logwatch/scripts/services/ folder. Alternatively you could of course create a symlink to your file.
2. Create the configuration file for the log group
Well, this was not exactly obvious, but you need to create a configuration file under /etc/logwatch/conf/logfiles/. Let’s say we create something like my.conf. Note that you will need the NAME of this file in the next step. This file can contain something like
# Place this file to the /etc/logwatch/conf/logfiles/
LogFile = /var/log/my/my.log
Archive = /var/log/my/my.log.*.gz
*RemoveHeaders
Here you put the ACTUAL log file and archive names. So you let logwatch take care of giving you the right input. This is already better than doing it all yourself isn’t it?
For one log file creating a &quit;group” may sound an overkill, but this is how it works and it is OK if you start reusing stuff, but for now this is a “necessary evil”.
2. Create the configuration file for the service
Ok, now we create the file which will be picked up by the logwatch and determine which script to run (by looking up the name!). So we create my-service.conf under /etc/logwatch/conf/services/ and put something like
# Place this file to /etc/logwatch/conf/services/
Title = "MY Service"
LogFile = my
Note that here you use the LogFile directive to refer to a LOG FILE GROUP, which was very confusing to start with, but OK, “was there, done that”.
Finally, you can do something like
# sudo /etc/init.d/0logwatch
and check the corresponding account mailbox for the result.
I hope this spares somebody time…
If you wonder why I’ve spent so many lines above in this post and haven’t referred to the Zabbix documentation the answer is rather simple. The Zabbix documentation on version 1.8 is very concise and states exactly two words: “Email notification.”.