Cloud Server Forensics Take Center Stage

At Threat Stack, we’re constantly exploring ways to advance cloud server forensics. We’re especially attentive to this as it’s an area of cloud security that’s becoming more critical since the attack vector of cloud is growing.

Forensic logs can lay out the scope of an attack that’s occurred on your servers, but getting to the bottom of what’s been done is usually much easier said than done. In fact, you can easily find yourself paying up to $600/hr for a security consultant to do this exact work if you don’t have the right tools in the first place. But what does it mean to have the right tools?

Do existing methods work?

You can assume that your prevention methods are so mature that you won’t ever need to do forensics, but that’s a big risk. Even if you think your traditional threat prevention methods really are solid, it’s wise to assume that a breach will — not might — happen.

We found that companies using the cloud have attempted to use traditional forensics solutions to solve their cloud forensics needs. Digging deeper, we discovered that these solutions failed them in the end because their systems do not consider the transient nature of data in the cloud (e.g., what happens if the server is destroyed before you have been able to collect your evidence?) Unless you are savvy enough to save your disk images before you deprovision your cloud boxes (AND assuming your IaaS provider supports that — many of them do not), you will not be able to successfully determine what happened on a box using traditional forensic disk acquisition and analysis tools like Encase. Moreover, these methods are time-consuming and require the skills of an expert.

If you’re a typical cloud user, you may not have these experts on-hand — this is where you shell out lots of money for consulting services.

Detecting all of the things (even internally)

You know your business needs a dedicated solution for understanding exactly what happened when a breach occurred — and ideally one that requires little to no human intervention so that you can continue focusing on other security priorities.

Let’s say you have a rogue employee running malicious code on your own servers right this moment. How would you go about discovering that this is happening? Well, it could either cost you $600/hr for a security consultant to come in after-the-fact to discover this compromise (and likely be late to perform effective remediation before valuable data is stolen), or you could be proactive and receive alerts as soon as abnormal activity occurs.

Let’s use ‘Bob’ as an example. Bob is a disgruntled employee and has decided to maliciously install a backdoor on your system. Using traditional methods, your sysadmin may never know he’s doing this.

Using Cloud Sight, your sysadmin will get notified of this strange activity through an alert:

They can quickly gather more information about this strange activity by clicking on the ‘Process Details’:

Let the forensic investigation begin.

The details shows that nc was launched from ‘/sbin/service1’:

A search in Cloud Sight for ‘service1’ shows activity from Bob around /sbin/service1:

Pivoting in ‘Process Details’ again, your sysadmin will see that Bob was creating this backdoor and attempting to cover his tracks by deleting evidence of his activity in local system logs (by removing ~/.bash_history). But since Cloud Sight forensics are stored off-box, everything is recorded!

What can we learn from this?

Well, first, never discount the fact that attacks can originate from inside your own company by someone with good credentials. On top of that, having a forensic bread crumb trail to detect both external and internal activity can seriously reduce — or even eliminate — expensive expert consultants to identify what happened. Since we record and archive all processes and network activity, you won’t have to fork up thousands (and even hundreds of thousands) of dollars for an outside source to investigate your logs.

On top of that, instantaneous detection of a compromise, along with complete forensics history in one-click, will lead to rapid remediation of vulnerable systems and data. A little cloud security monitoring and proactive evidence gathering can go a long way towards saving yourself from a lot of hassle.

The heart of Threat Stack's innovation and product vision, Jen is passionate about designing the next generation of security tools for our modern infrastructure. Before co-founding Threat Stack, Jen worked on Mandiant's Research and Development team under Jamie Butler, developing innovative tools to analyze malware. At Symantec, she worked on the development team architecting a backend, that processed tens of terabytes of security data every day, as well as a security analyst, in Symantec's Managed Security Services division aimed at protecting the Fortune 500 against threats.