Mutexes and malware analysis

There is many way to investigate malware and to “find evil” in an unknown executable. There will be situation where looking at a list of running processes won’t give you information to raise a red flag. You can always go deeper and perform more manual analysis. Even though this might be fun it might also be very time consuming…and we don’t always have the luxury of time neither the resources.

Analysis of mutexes (sometime called mutant) can be a pretty good way to continue your analysis and find more evidence of “evil”.

Mutexes

A mutex object is a synchronisation object whose state is set to signaled when it is not owned by any thread, and non signaled when it is owned. Only one thread at a time can own a mutex object, whose name comes from the fact that is it useful in coordinating mutually exclusive access to a shared resource.

Continuing reading the MSDN documentation:

For example, to prevent two threads from writing to shared memory at the same time, each thread waits for ownership of a mutex object before executing the code that accesses the memory. After writing to the shared memory, the thread releases the mutex object.

There is also two types of mutexes:

local mutexes: those are unnamed mutexes. This type of mutex exists only within the process of the application.

system mutexes: those are named system mutexes. This type of mutex is visbile throughout the operating system, and can be used to synchronise the activities of processes.

So what does that mean? Well if you have two threads that must access a memory structure, but only one can safely access it at a time, a mutex can be used to control the access. The second questions is: what’s the relation with malware analysis? Well malware are a little bit selfish and they don’t like when two instances of the same malware run at the same time. Mutexes are therefore the way to go and sometimes malware create mutexes with specific name (e.g. hardcoded name) to ensure that only one single instance of the malware is actually running.

On a low level basis, a thread gains access to a mutex with a call to WaitForSingleObject and any subsequent threads attempting to gain acess to it must wait. When a thread is finished using a mutex, it uses the ReleaseMutex function. A mutex can be created with the CreateMutex function. One process can get a handle to another process’s mutex by using the OpenMutex call. As said before, a malware will commonly create a mutex and attempt to open an existing mutex with the same name to ensure that only one version of the malware is running at a time.

First hint: if you are doing malware reverse engineering keep looking for those functions…they will be a pretty good indicator!

Where to find the mutexes?

On a live system
If the system you are investigating is still up and running, you can use tool like ProcessHackerIn a memory dump
If you are using a tool like volatility you can use the pluging mutantscan, this will output all the mutexes (use the -s option to hide the mutexes with no name) . Example from the GrrCon 2012 DFIR challenge (Attention: this link will give the answer to the challenge):

If you use a tool like Mandiant Redline, it will flag suspicious mutexes and give the process a higher Malware Risk Index (MRI):

In an executable

A few solutions here:

You can extract the strings of the executable. You might find something but this might not be the most accurate way of doing it. Also this might not always work but you can give it a try.

You can do some reverse engineering on the executable. Look for the functions mentioned at the beginning.

Use a sandbox to run the malware (virtual lab, cuckoo, etc.).

That’s great but how can I flag a suspicious mutexes?

Well that’s THE question! And there is no easy answers to this question. Your friends Google and Wikipedia are always a good source of information.