Qindel Group

How often have you been in the need to get some log statistics? I have seen many packages around, in particular those monitoring some specific internet applications (web, ftp, mail, …), everybody knows stats packages like analog and webalizer, if we are talking about web you can also use Google Analytics.

But more often than not, you always face one of the following problems:

• The log format is not the standard web log
• You need realtime info.

In the first case you end up writing your own parser of the log file (unless you have written the application, but then we would come on which are the requirements of the application, and why was monitoring not included). We usually do that in perl, although I have seen parsers in many languages including shell, and even Java…

But in the second case, what you would need is an online parser which gets you statistics by parsing incrementally the log files.

Who has not used MRTG, Cacti, Cricket, or any of those RRD applications that allow you to see how well your Internet line is doing?

In our case we have come up with a little solution, which might seem quite hard at the beginning but that does the job quite well. We use the combination of 4 elements:

• Net-SNMP. This provides us with the tools to speak the SNMP protocol.
• SNMP MibProxy. This provides us a neat clean interface to the snmpd daemon.
• SNMP Logparser. This is our online and incremental log parser build template.
• Cacti. We use cacti to display graphs, although we have used others like opennms, cricket, mrtg, (or other non opensource software).

Ok, now we have these 4 different tools and the question is how we build them together? I will not get into the specific details, please use the man page of snmpd, the documentation of cacti and the POD of the specific perl modules for that, but I will try to highlight which are the main steps of getting this right.

1. Create the MIB for the counters and stats that you want to monitor. I will not get into the details of this here. My quick recomendation is that you copy one of the existing mibs and change them to suit your own needs (in my computer you can find it in /usr/share/snmp/mibs/IF-MIB.txt). See among RFC1212 and others (http://www.ietf.org/rfc/rfc1902.txt).
2. Create the interface in snmpd with MibProxy. This is accomplished by adding the following line to snmpd.conf (obviously you need to change the OID):
   pass_persist .1.3.6.1.4.1.17171.1.8 /usr/local/bin/mibProxy

3. Test the interface with snmpget. For this you need to add a static value to the mib counters in /var/lib/mibProxy/logparser.properties
   myDescr.0=test2
   myCounter.0=21
   If you get your 21 from the command line “snmpget localhost myCounter.0″ then you are in the right track, if this doesn’t work, please check the man page for mibProxy.

4. Create a SNMP::LogparserDriver subclass which should implement the following methods: evalBegin, evalIterate, evalEnd. These methods will allow you to easily parse the info that you need and leave them in the former /var/lib/mibProxy/logparser.properties
5. Run the logparser script, which will invoke your specific class, from cron, every 5 minutes for example.
6. Graph the results in cacti via SNMP

Here are some of the things that we have used this for:

• Show the rate of new subscribers. Very useful for new campaign launches in marketing.
• Show the number of purchase transactions per seconds (including failed transactions)
• Get this nicely displayed on a handset, so that you always know how things are going
• plug this info to an NMS system, to alert whenever the number of transactions were lower than expected.
• Create dashboards which reflected the business as a whole. Just to note that some of them were used as much by the Technology guys as well as the marketing guys, providing a real good team building, and common objectives…

I hope you enjoyed this one as well.