Friday, October 12, 2012

I was writing to announce the last week of the month of Volatility plugins
is finished, and we now have five more in-depth blog posts covering
Windows
and Linux internals and rootkit detection. These have all been posted
on
the Volatility Labs blog.

Post 1: Detecting Malware with GDI Timers and Callbacks

This posts covers analyzing malware samples that use timer callbacks to schedule actions.

Friday, September 28, 2012

I was writing to announce that week 3 of the month of Volatility plugins
is finished, and we now have five more in-depth blog posts covering Windows
and Linux internals and rootkit detection as well as a bonus plugin that analyzes Internet Explorer browsing history. These have all been posted on
the Volatility Labs blog.

Post 1: Detecting Malware Hooks in the Windows GUI Subsystem

This Windows focused post covers detecting malware hooks in the Windows GUI subsystem, including message hooks and event hooks, and what effects these hooks can have on a compromised system.

This Windows focused post covers finding and recovering shellbags from memory, the forensics importance of shellbags, and analyzes the effects of anti-forensics on shellbag timestamps. It concludes with covering the traces left in shellbags by TrueCrypt.

This Windows focused post introduces two new plugins, one named gahti that determines the various different types of USER objects on a system and another named userhandles which traverses the handle table entries and associates them with the owning processes or threads

This Linux focused post analyzes the 2008 memory challenge with Volatility. It walks through the artifacts produced by the winning team and shows how to recover the same information with Volatility. It then shows plugins in Volatility that can recover artifacts not produced by the winning team.

Friday, September 21, 2012

I was writing to announce that week 2 of the month of Volatility plugins
is finished, and we now have five more in-depth blog posts covering Windows
and Linux internals and rootkit detection. These have all been posted to
the new Volatility Labs blog.

Friday, September 14, 2012

I was writing to announce that week 1 of the month of Volatility plugins is finished, and we now have five in-depth blog posts covering Windows and Linux internals and rootkit detection. These have all been posted to the new Volatility Labs blog.

Post 1: Logon Sessions, Processes, and Images

This Windows focused post covers linking processes to their logon session, detecting hidden processes using session structures, and determining the loaded the drivers mapped into each session.

Monday, August 13, 2012

In this blog post I will introduce a new Volatility Linux
plugin, tmpfs, and discuss its uses
and implementation. The purpose of this plugin, which can currently be found here, is to reconstruct any tmpfs
filesystem contained within a Linux memory capture and fully recover it to
disk.

What is tmpfs?

Tmpfs is a Linux filesystem whose
contents reside only in memory. This means that files and directories inside of
tmpfs mounts are never written to the local disk and that once a tmpfs mount is
unmounted, the entire filesystem is “gone”.

Why is tmpfs interesting?

Tmpfs is interesting from a forensics perspective for a few
reasons. The first is that, in a traditional forensics scenario, the
investigator expects that he can shut a computer off, images its disk(s), and
get back the filesystem at the time of when the computer was running. With
tmpfs, this is obviously not true…

The second reason that tmpfs is interesting is because of
how it is used by Linux distributions as well as by attackers and certain
malware samples.On many Linux
distributions, the /tmp directory is
mounted as tmpfs so that the directory will be cleared on reboot. This helps
speedup the boot process as the init code does not need to unlink every file
and directory under /tmp, but it also means that the traditional (disk only) preservation
process will miss any files stored in /tmp.
This directory often contains scratch data and install files for many
applications as well as for rootkits/malware that try to be agnostic to the
distribution they are being installed on.

Linux also uses tmpfs to
implement shared memory through /dev/shm.
While recovery of this directory may help in recovering IPC data, its main
purpose related to forensics & IR is that it is often used as a scratch
directory by attackers to download files, compile programs, and to store the
output of commands and malware hooks.Since the filesystem goes away on reboot, attackers know they can
frustrate and/or defeat traditional forensics investigations using this method.

The third reason that tmpfs is of interest is that Linux
live CDs use tmpfs as the underlying store for all changes made to the
filesystem after the machine boots, since obviously it cannot write back to the
CD. If you are interested in reading about memory analysis of live CDs, I did a
presentation and white paper on it at Blackhat DC 2011 ( slides | paper ) so I will not
recap it all here.The paper also
discusses much of the kernel internals described at the end of this blog post.

Finally, while still being researched, it has been
discovered that many Android applications use a tmpfs instance created by the
Android runtime to store “interesting” runtime data and state.The tmpfs plugin has already been tested on a
number of phones using Volatility’s Android support and the data was verified
to be fully and correctly recovered on all devices.For the latest details on this, please
contact Joe Sylve (@jtsylve).

How do you use the plugin?

The plugin requires two steps to work. First, you run it
with the ‘-L’ option to list the tmpfs filesystems within the image:

When this command is finished, the entire filesystem of
“/dev/shm”, including all files, directories, and metadata (atime, mtime,
permissions), will be replicated to the “outputdir” directory. In a real forensics investigation, “outputdir”
should be on an external drive or a separate mount point that can be
remounted read-only after the filesystem is recovered.

Limitations

When recovering the filesystem from memory to a disk there
are a few limitations that investigators should be aware of:

1) While the script does preserve the modified and accessed
times of the recovered files and directories, it has no method in which to
preserve the create times. This is because the Python os.utime function only supports modifying the accessed and modified
times. In order to change the create times on a standard Linux filesystem, the
script would need to interact with something along the lines of debugfs,
which can be error-prone and only supports the Ext family of filesystems.Similar burdens are faced on Windows and Mac
filesystems as well.

The result of this limitation is that the create time of all
entries in the filesystem will actually be the time that the Volatility plugin
created them. If enough interest is shown in the plugin and this limitation, I
can easily extend the plugin to support writing out a mactime file with all the
correct information.

2) The owner/group information is not preserved for
recovered files. This may be added to future versions of the script, at least
if the output filesystem is Linux based.

3) There is currently no support for recovering deleted
files within tmpfs stores.

How does the plugin work?

Now that we have explained the purpose of the plugin, it is
time to dig into some Linux kernel internal in order to explain the plugin's
operation. If you are not a programmer or not interested in kernel internals,
you can freely skip to the end of the post.

For the plugin to work it needs to be able to the following:

1) Find and list all tmpfs mount points in the memory image

2) Recursively enumerate and reconstruct the entire
filesystem for every tmpfs instance

3) Record the needed metadata of each file and directory and
update the recovered files on disk to match it

4) For every file found, recover the file contents and write
them to disk

1) Finding all tmpfs mount points

The tmpfs mount points are found by leveraging the
capabilities of the mount plugin (volatility/plugins/linux/mount.py).
The mount plugin works by walking the mount_hashtable
hash table, which stores a vfsmount structure for every mount
point. This is the same operation the kernel does at runtime to populate the /proc/mounts file.The tmpfs
plugin then filters the vfsmount
structures to only those representing tmpfs mounts.This is all that is needed to implement the
‘-L’ option of the plugin.

2) Recurse the filesystem

Each vfsmount
structure contains a pointer to the root directory entry of the filesystem, of
type dentry, in its mnt_root member. dentry structures are generic structures used to represent files,
directories, pipes, sockets, and all other file types within the kernel. They
have no 1:1 mapping on disk and there can be multiple dentry structures per inode.

Once the root dentry structure
is obtained, we can then recursively traverse the entire filesystem by walking
each dentry’s d_u.d_child member. For each entry in the child list, we check if
it’s a directory or a file. If it’s a directory, the plugin simply creates the
directory in the output directory’s hierarchy and then recursively processes
it. If it’s a file, then we need to recover its contents from memory and create
the file in the output hierarchy.For
both directories and files, we need to update the metadata of the newly created
files to match those in memory. This process effectively recovers the entire
filesystem to disk in the exact order and layout as in memory.

3) Recovering metadata

Recovering metadata is a simple process because all the
information is directly stored within the inode
structure of the file or directory, which is stored in the d_inode member of the dentry structure.This line of code from the plugin accesses
all the metadata from the inode that
is currently recovered:

In
order to recover a particular file’s contents, we need to be able to locate the
structures that track its memory pages within the kernel’s page cache and then
find the actual physical pages that store the contents.

The
page cache is a performance enhancing
mechanism used by the kernel to cache physical pages in RAM so that they do not
have to be read from the raw device or network device on which they are stored each
time they are accessed. For example, on an active SSH server, a number of files
(passwd, shadow, sshd_config, /bin/bash, etc) are going to be constantly read
as users log in and off the system. If the kernel had to fully read in these
files from disk to memory each time a user logged in, performance would be
unacceptably bad.To alleviate this,
pages that are read in from disk are then put in the page cache, so the next
time an applications needs to be executed or reads a file, the kernel can simply
return the page from the cache without having to touch the disk. An aside: If
you have ever dd’ed a disk image in Linux and watched ‘top’, you may have noticed
that your entire RAM contents was being used inside the kernel. Much of that
was the kernel reading ahead on the disk being imaged and filling the page
cache with the next blocks to be processed by dd.

Since
the filesystem type we want to recover (tmpfs) is stored only in memory, it is
implemented so that all its pages are always stored within the page cache. This
lets the filesystem work as all others do, and the kernel will never look for a
disk or other source to read from because the files will always be up-to-date
within the cache.

In order to locate the
pages of a particular file, we need to use its index into the page cache. This
is stored within the dentry.d_inode.i_mapping
member which is of type address_space.address_space
structures are used to track sets of sparse physical pages that together
form a contiguous data unit (in this case, a file within a tmpfs mount).To access the pages of a file inside its address_space, we need to walk the radix
tree of struct pages stored within
the address_space’s page_tree member. A struct page tracks a physical page and lets the kernel determine
the physical address of the page’s contents.

To
find the correct index in the tree for a page, we simply divide the offset of the
page into the file by page size (4k in this case). We do this in a loop for the
entire file in order to recover each page sequentially.I will save the reader the pain of having to
understand the kernel’s radix tree, but the interested reader can look at the radix_tree_lookup_slot function in the
tmpfs plugin. The page cache and its storage format is also very nicely explained in Understanding the Linux Kernel 3rd Edition. Once we are able to
walk the page_tree of a particular
file and recover each index (a struct
page), we then need to determine the physical address of the page it
tracks.

Finding
the physical address is performed by indexing the struct page into the mem_map array.
This array holds a struct page for every physical page of memory. Indexing into mem_map will give us the page number of the physical page, which we can then shift
left by PAGE_SHIFT (12) in order to get the page’s physical offset. This process roughly
corresponds to the page_to_pfn macro
within the kernel.

Once
we have the physical offset, all we have to do is read it using Volatility’s
API and then concatenate each page’s contents together into the file. At this
point we will have recovered the entire filesystem along with its contents and
metadata.

Conclusion

This blog post has showcased a new and exciting Volatility
plugin. Being able to fully recover tmpfs filesystems out of memory greatly
adds to the depth of Volatility’s Linux support for both traditional Linux
installs as well as Android devices.By
incorporating this plugin into your forensics and incident response processes,
you will recover information that until now has large been ignored or missing.

Please check back frequently as I will be show casing more
plugins leading up to the release of Volatility 2.2 at the OMFW Conference. If you are interested in the latest research in memory forensics, I highly
suggest you register for and attend OMFW as many of the best memory forensics
researchers will be presenting and attending.

If you have any questions or comments about this post, please Email me, catch me on Twitter (@attrc), or reply in the comments section.

Friday, June 29, 2012

I am writing to announce that Volatility now supports captures from Mac systems! I gave a talk on the new capabilities at the recent SANS DFIR Summit. The presentation can be found here. Complete information on how to access the branch and create profiles can be found here:

The wiki page will be updated as user-visible changes are made to the branch. Otherwise, you should check back here often as many more plugins and analysis features will be released in the coming months.

If you have any questions you can find me on Twitter (@attrc), leave a comment on the blog, or shoot me an Email.

Wednesday, June 20, 2012

A week from today I will be speaking at the SANS DFIR Summit about the research and development I performed to add Mac OS X support to Volatility. The proliferation of Macs for both business and personal use is well known, and investigators will be increasingly seeing them during their course of work.

While there are a number of people who have analyzed Mac's on-disk artifacts, including upcoming talks at the summit by Sarah Edwards, there has been little documented work covering the in-memory data structures and algorithms. Previous work by Matthew Suiche as well as by the Volafox team have covered the beginnings of Mac memory analysis, but both stopped short of the full coverage needed for deep investigations.

During the presentation, I will be discussing the types of artifacts recoverable through Volatility's Mac support, such as process listings, memory maps, loaded kernel extensions, network connections, and also some Mac-specific constructs such as the I/O Registry. The new Mac support also includes the ability to handle both 32 and 64 bit Mac memory reader captures, and I will be discussing this as well as how to use Mac Memory reader during investigations. I will conclude the talk by going over some interesting kernel-level Mac rootkits that alter dynamic data structures and discuss how Volatility can be used to detect them.

Since everyone in the audience will not be a programmer and/or expert on operating systems internals, I have abstracted some of the details away, but a light dive into kernel internals is inevitable when dealing with kernel memory analysis.

After the talk, the source code for all of the current Mac support and analysis plugins will be available within the Volatility SVN repository. People will then be able to use the functionality themselves as well as provide testing of the new features. The Mac support is under active development and I expect many new features to be added soon as well as stabilizing of the existing source code.

If you have any questions or comments before the talk, please send an Email or reply in the comments.

For those attending the first day of the conference, I highly suggest you check out the talk by Joe Sylve as he will be discussing acquiring memory from Android devices and then subsequent analysis with Volatility.

Monday, June 4, 2012

Hello and welcome to my new blog where I will be posting about my research into memory and disk forensics as well as other related topics. For those who do not know me, my name is Andrew Case, I am active on twitter (@attrc), and I have a collection of my past works and speaking engagements organized on my personal website: http://www.memoryanalysis.net/