Lately I have been toying with rewriting the Flowview plugin in Cacti with something that is a bit more usable. The current plugin has some major disadvantages. Namely it relies on the flow-tools package to do all the heavy lifting such as collecting flows, parsing everything, creating reports. In reality, the current plugin is really more of a viewer for flow-tools. Flow-tools itself has several major drawbacks. It doesn’t support Netflow v9 or v10 (IPFIX). The last one is a major problem considering that VMWare 5.1 has moved all of its distributed switches to only outputting v10. I also have to rely on the flow-reports command to spit out a few canned reports, so its not very customizable. It also has major issues when receiving flows from routers that are in different time zones (lots of wackiness when saving the flows per hour, and attempting to use those directories for reports).

To rewrite Flowview. There will be 3 major parts. The Collector (and storage), the Parser, and the Interface. The Collector will be as small and efficient as possible to help it collect large amounts of incoming netflow packets. Spawning multiple threads will help a bit too I think. I bet my friend Greg can help me out with a large stream of Netflow packets to stress test it with. I will be storing the flows in a MySQL database (partitioned per day). As with the Syslog plugin, we have had pretty good luck with performance utilizing MariaDB 5.5 partitioned tables with large data sets. I will be directly inputting into a Memory table (again to help improve performance with the Collector), and will do some processing on those entries (via the Cacti Poller) every minute to move them into long term storage. The UI will be easy enough, and the Parser will be doing the heavy lifting of querying the database. I want to move to a queued job approach of querying for reports, so that we can get a better handle on the load that each report can put on a server.

The first thing to write is the Collector. The easiest Netflow packet to decode is v5. It has completely static length headers, flow record sizes, and data field lengths. You can take a quick glace at the format and fields over at the Cisco site.NetFlow Export Datagram Format
Each packet you receive will first contain a header telling you a little about the included flows. Following that will be each flow record. You can have multiple flow records in a single packet. A simple loop and a few lines of code to unpack everything is all that is really needed.

Now, for anyone wanting to write their own Netflow parser in PHP, I have included some basic code below to get you started. From here you will want to expand it to do validations, possibly some logging, and decode more Netflow versions. I will show how to decode v10 (IPFIX) in my next blog post.

PHP

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

<?php

# We first setup our listening UDP Socket. We use the default port of 2055, but that can be changed to anything

Today, the long awaited release of CactiEZ v0.7 is out. While the release took much longer than anticipated (I reworked it several times) I believe the final product turned out better because of it.

There are several key things to note about the new CD.

OS is CentOS 6 x64 (there is no 32bit version)

Now prompts for a few key items during the installation such as Network info, timezone, and password

More secure out of the box than the previous builds (some linux hardening, randomized mysql passwords, etc…)

PHP Timezone is automatically configured for the timezone entered during install

Cacti and all plugins are now installed via RPMs from my custom repository for easy updating of Cacti

Switched to RSyslog instead of Syslog-ng

NTop removed

Now includes a custom setup plugin that walks you through setting up your Cacti server (plugin installs, templates, etc…)

Custom jquery based theme donated by an anonymous source (Thanks!)

No upgrade path from CactiEZ v0.6 provided

It is currently available for download via torrent at CactiEZ. EDIT: I have now added a HTTP download link.

I would like to thank all the beta testers that have helped with this project. I had 500+ beta requests so congrats to those who were selected and got an early peak at it. For the 1000 other people that asked me why it took so long. Please realize that every minor change that occurred would require testing that consisted of rebuilding the RPMs (if necessary), recompiling the CD, transferring the CD, re-installing from CD, and then actually starting the testing after all that was done. All I can say is thanks to VMWare ESXi and a lot of custom scripting, I was able to stream line the process.

For anyone that appreciates the hard work that goes into a project like this, I have provided a Donations link on the CactiEZ page. I also love Amazon Gift Cards.

For support / questions, please use this thread over at cacti.net – CactiEZ Support

Update: I would like to thank everyone that has made a donation no matter how small. I shall drink a pint in your honor while I decide what to put in the next CactiEZ (snmp traps maybe?)

For anyone interested, I will be speaking at the Open Source System Management Conference 2012 in Bolzano, Italy on May 10th, 2012. If you would like to attend, please check it out below. There is going to be a lot of great speakers and informative lectures going on.

Save the date!

Thursday, May 10th - Bolzano / Italy - 9.00 a.m.

International keynote speakers Lively discussions Shared experiences

The yearly Open Source System Management Conference goes in its 4th edition. The event over the past years has grown to one of the most important meets in Europe for

IT managers, system administrators and everyone interested in Open Source monitoring solutions like Nagios, Cacti, Shinken or Icinga. Join this free event and get in touch with international experts presenting relevant trends, new strategies, actual projects
and practical user experiences.

I recently had the idea to write an agent for a few windows servers to execute some custom commands so I could graph the data in Cacti. I have dabbled in making my own Windows SNMP extension agents in the past (and I love SNMP) and while they work well; I really didn’t want anything that complicated.

Now not being someone to re-invent the wheel unless I have to, I started looking around at a few other existing agents. A few of the requirements I had in mind were that I didn’t want the communication in plain text, it needs to be somewhat secure (password or ACL), it needs to be fairly easy to install and configure custom commands, and most importantly I need to be able to easily communicate with the agent over PHP. After finding agent after agent that did 1 or 2 of my requirements, I settled on the Nagios NRPE Agent. While it does have some quirks, it does satisfy the first 3 of my requirements, so all I would have to work out is the 4th. The NRPE Agent can use SSL to communicate between the monitoring server and the agent. While it does not have the ability to require a password, it does allow me to set an ACL on the agent to specify the IP addresses allowed to communicate to it. Configuration is just editing a text config file to add my custom commands, which is really simple. It also makes it pretty easy to push any new updates to my servers.

As for the 4th requirement, communicating with the agent from PHP. I wanted to do this, so I could write some Cacti Script Server scripts, and not have to launch a new process to then query the agent. For NRPE, you normally would have to run the check_nrpe executable with the command line arguments for the command. This was messy and an extra step which would add some latency and slow down my polling. I looked around, and really couldn’t find much info on the protocol NRPE uses between the client and host. I did finally manage to find a perl script that did basically what I wanted, or at least laid out the basics of the required packet for me, so I didn’t have to resort to Wireshark to figure it out. The only thing required was to figure out exactly what it was doing, and then implement the same thing in PHP. The perl script was a few hundred lines of code, and had 4-5 required libraries, which was something I was going to have to do away with.

A few of the tidbits I gleamed. I first made the mistake of attempting to use fsockopen to create the SSL connection, but for SSL to work with NRPE, it seems to require the use of the ADH cipher. Just trying to do a straight up SSL2 or SSL3 connection is going to fail with an error about the cipher (check the agent log). This didn’t pose much of a problem though, since PHP also has lots of nice built-in functions for dealing with SSL streams, and its really easy to change the cipher used with those functions. This also makes it easy, since I really don’t have to deal with certificates or anything else you would normally need to do with SSL. The next thing is that the NRPE agent expects the packet to always be the exact same size no matter the command. We are basically going to have to pad the command with nulls to be 1024 in length, we then add all our CRC checksums and other misc headers to this. Overall the packet itself is very simplistic, and doesn’t include anything really complicated to create.

Anyway, on to some code. This is the simple script I wrote to test querying the agent.

Overall it took very little code to create the communication which made me extremely pleased. It turned out to much much smaller than the perl code I was originally working with. This will ofcourse have to be modified to conform to the Cacti Script Server usage, but that is really the easy part. Once I get a few templates completed (for the normal NRPE commands) and I will post them for everyone to use.