Event log management made easy

Most companies' log files carry too much noise. For a better log management system, turn the normal model on its head

InfoWorld|May 22, 2012

When it comes to log management, most administrators follow the traditional route of generating all possible events that might need to be captured, then choosing which events should create actionable alerts. The centralized log management system then picks up, centralizes, and correlates these entries. The result is information overload, much akin to the log entries generated by the average firewall (you all know how much I love firewalls).

No one reads their firewall logs because they're full of information that doesn't need to be acted upon. Mostly, the logs record legitimate traffic erroneously reaching the firewall, such as network broadcasts or harmless exploratory packets. Alternatively, they track blocked port probe packets that take too much energy to research. But if you never check your logs or make actionable events from them, why collect them in the first place?

[ There's gold in your log files, and Roger A. Grimes will help you find it. | InfoWorld's Log Analysis Deep Dive PDF special report shows how effective collection and analysis of log files can help you improve security, troubleshooting, compliance, and systems management. | Stay up to date on the latest security developments with InfoWorld's Security Central newsletter. ]

The way to make an event log management system that others will be jealous of is to collect only events that generate actionable responses. If you or your response team won't immediately respond to an event log entry, then don't generate it. The result: no noise, no false positives.

The key is in defining all the events that you know, for sure, are anomalous. Instead of trying to decide on what is actionable among all the noise, turn the model on its head and generate the opposite. Here some examples:

If your Administrator account is renamed, alert on any attempted log-ons as Administrator.

If server A should never connect to Server B, alert when they connect to each other.

If normal network traffic between point A and point B is 1GB per day, but it unexpectedly jumps to 100GB, research it.

If your company never sends files to China, then suddenly hundreds of gigabits of data head there, check that out.

If you run RDP on a nondefault port (say, port 63389 instead of 3389), alert when someone connects on 3389.

If employees in factory location A never do work at night, alert when a user logs on at midnight.

If you have no members in your Domain Admins group and someone adds him- or herself, check that out immediately.