Tips and Trends on Database Log Management

Buried deep within enterprise IT infrastructures, databases can be said to hold the “crown jewels” of an organization. Unfortunately, database security is often lacking, leaving sensitive, business-critical information such as customer data, financial details and more vulnerable to hackers. Department of VA, TJ Maxx, TD Ameritrade – these are just a few of the many organizations that have driven the media wild over data security breaches in the last year.

It is common that database administrators (DBAs) are assigned the task of database security, but this is an issue that should be of utmost importance to any business that wants to stay in business. TJ Maxx reported at least 45.7 million credit and debit card numbers stolen over a period of several years, costing the company an estimated $168 million.1 Proper security measures may not have stopped the initial hack-in, but perpetual data theft could have been avoided through careful log collection and analysis. This article will not only discuss the importance, challenges and benefits to database logging, but will also offer a few forward-looking trends to managing your database logs.

About Logs and Database Logging

Databases are now becoming one of the most voluminous log generators in the enterprise – rivaling firewalls for the top spot. Most databases (i.e., Oracle, Microsoft SQL Server, IBM DB2, MySQL, etc.) will log system starts, stops and restarts by default, but database logging isn’t merely about keeping the system running, particularly when your databases contain sensitive, private information. Security and compliance requirements must therefore be considered when configuring your database and managing your logs. In fact, regulations such as PCI, HIPAA, and FISMA all mandate log monitoring, with Sarbanes-Oxley strongly recommending it as a best practice.

Database logging thereby becomes an essential (and required) component of database security – and it makes sense to not only focus on “keeping the bad guys out,” but also to take a “what’s going on in here?” approach. After all, you may not know who the “bad guys” are. Logs can provide a continuous fingerprint of everything that happens in your IT systems and with your data and will point you to the “who, what, when, where” information of any breach – whether the malicious behavior comes from outside hackers, a disgruntled employee or another source.

Database security is a task often assigned to DBAs, not because they’re security experts, but because they know the ins and outs of databases. If configured properly, databases may be logging overwhelming amounts of files, perhaps up to gigabytes of data per day. Typical database log events may include:

User logins and logouts; Database system starts, stops and restarts; Various system failures and errors; User privilege changes; Database structure (metadata) changes; Most other DBA actions; and Select or all database data access (if configured to be so).As we know, hackers are always looking for new ways to break through security barriers to access your sensitive information, and all preventative security measures fail at some point. Thus, because you are not able to guard against every malicious hacker, logs will at least allow you to detect such security breaches as well as actually figure out how it was done during the incident investigation. At a minimal level, logs must be collected and archived, but log analysis makes the data significantly more useful. In more explicit terms, log monitoring and management should include:

Collection: Gathering log data where it is being generated via an agent or remotely, Transfer: Securely transmitting log data to a central server for analysis and storage, Alerts: Issuing real-time alerts to database administrators if needed, Reporting and analysis: Providing reports and analytics based on log data, and Storage: Securely storing logs as long as prescribed by your retention policy and then, just as safely, destroying them.The above examples for managing your log data will help you keep tabs on the activities occurring in your business. Regularly collecting log data is a best practice for incident response and can save you during crunch time after a server crash, data theft or surprise visit by your friendly auditor. Alternatively, if someone is downloading an entire table or changing a database schema while being logged on from a remote connection, a real-time alert will catch your attention. Further, reports may help you track and analyze login failures and successes or after-hours access to better evaluate insider privilege abuse. In other words, database logs can help you catch unusual behavior before a problem gets out of hand and into headline news.

Database Log Management: Where to Begin

If you are just beginning to set up a method for managing your database log data, be ready for a large volume of log records as well as issues pertaining to log availability and log format complexity. Log formats can be verbose and obscure, particularly in cases where a single message spans multiple lines, making it difficult to extract useful, actionable information via automated tools.

Other challenges to database logging include decreased performance and storage restrictions. Unlike other situations where logging has a minimal impact on system performance, database audit logging slows down the database, sometimes significantly. High-performance databases are built to provide thousands of data transactions per second; logging all of these presents a challenge to system IO as well as CPU and disk storage resources. Since many regulations specify a three to 12 month period for log retention, plus a longer period for log retention on tape or another dedicated storage tool, database logging is typically getting a bad rap among DBAs already spread thin for time and resources.

Because the difficulties associated with database log management can seem overwhelming, it’s best to take things one step at a time. Start slowly and build up your system from there. You’ll want to collect logs from multiple servers at one central location to facilitate incident analysis and response. This will also help prevent loss of log data during routine log rotations. To gain insight into what’s going on internally, conduct periodic reviews of DBA activity logs – you can then keep tabs on people entrusted with sensitive and/or private information such as customer data or product inventory information.

When beginning to organize and manage database log data, also keep in mind that manual log analysis can cost a lot in terms of time, human resources, etc. Popular database solutions such as Oracle, Microsoft SQL Server, MySQL, and more, tend to offer various basic logging options, but none comprehensive enough to really capture a continuous feed of database activity. By contrast, an automated log management tool will not only free up DBA time for other important database performance and security tasks but can also be more reliable and efficient than manually managing log data.

Further, with an automated log management and intelligence (LMI) tool, you can schedule log collection to occur at off hours when other database service operations, such as backups, are happening so that database performance is undeterred during the workday.

Trends in Database Log Management

Database logging often presents a new frontier for many security practitioners – one that must be conquered. Given that historically many databases were running without any data access and data change logging disabled (as by default), the key trend is that this is finally beginning to change. Why is it happening? There are two main drivers for this trend: PCI DSS compliance requirements that apply to those who handle credit card data and the proliferation of data breaches and data loss discussed above. The cost of data loss investigations in the absence of detailed access logs is absurdly high!

What would happen next? As more people enable logging, the challenge of what to do with all that data will emerge. Handling log storage and controlling log retention so that logs will be there when needed for the investigations will be the next trend.

After that, database log analysis will become all the rage. Analyzing logs for anomalies, suspicious user activities, unsafe administrator actions, privilege abuse as well as good old hacking will require deployment of log management tools with database-specific intelligence.

Further, logging guidance from IT best practices such as ITIL and ISO will become the norm, and we will reach the database logging nirvana, when logging is enabled because “it is the right thing” and not only due to compliance pressures or the latest data breach. Log collection and automated analysis will become the norm as new LMI technologies simplify managing the log flood as well as the process for making sense of logs.

So, enabling logging is a good start, and taking small progressive steps toward more in-depth log analysis can be greatly enhanced by deploying an LMI platform.

LMI tools are becoming increasingly advanced and are now typically able to take a deep dive into log analysis and management. A good log management solution will not only automate log data analysis, but also the whole lifecycle of log management.

There is also an increasing trend in logging across the entire IT infrastructure – firewalls, servers, network devices, applications and operating systems all produce logs – and can be managed by an LMI platform. In other words, jump on the logging bandwagon with your other system administrators and balance your IT infrastructure with a log management system that will work across multiple database servers, across various database types and with all other log producers in your organization. You will not only improve performance and business continuity, but also be able to put database log data in the context of other organizational log data to correlate IT activities with events occurring in your business.

Database log management is becoming a best practice for database security – you should be aware of who is accessing or changing your data, when they are accessing it and where they are accessing it. Luckily, you can combine database log management with other similar projects (such as firewall or UNIX server syslog management) and use a single automated LMI platform to enable efficient and reliable log collection, reporting analysis and retention. As long as you grow your LMI deployment in phases rather than trying to cover all logs on day one, you will be on the path to overall greater IT security within your organization.