Troubleshooting Memory Leaks With Just a Dump

Hello debuggers, the debug ninja is back again.Sometimes we have a scenario where a process is using a lot of memory, and the only data we are able to get at the moment is a user dump.Ordinarily data from tools such as umdh or xperf would be preferable because they provide memory usage data over a period of time and can include call stack information.However, umdh requires restarting the process (which loses the state of high memory usage), and xperf requires the installation of the Windows Performance Toolkit which may not always be an immediate option.

When we have such a dump we may not be able to specifically identify what piece of code is generating the high memory usage, but we may be able to narrow the scope of troubleshooting to a specific dll.

The first thing we need to do is identify what type of memory is using most of the address space.The debugger command !address –summary allows us to do this:

0:000> !address -summary

— Usage Summary —————- RgnCount ———– Total Size ——– %ofBusy %ofTotal

Free4897fe`6ff5a000 (7.994 Tb)99.92%

Heap90941`75ed1000 (5.843 Gb)93.47%0.07%

<unknown>2750`12e41000 ( 302.254 Mb)4.72%0.00%

Image9370`05a6a000 (90.414 Mb)1.41%0.00%

Stack1380`01700000 (23.000 Mb)0.36%0.00%

Other140`001bd000 (1.738 Mb)0.03%0.00%

TEB460`0005c000 ( 368.000 kb)0.01%0.00%

PEB10`00001000 (4.000 kb)0.00%0.00%

From this example we can see that most of the memory is used by heap.A process will usually have multiple heaps, each created by a call to HeapCreate.We can examine the size of each of these heaps with !heap –s:

From the above output we can see that most of the memory is being used by heap 00000000033a0000.

At this point we need to try to identify what this heap is used for.A brute force method to do this is to search memory with the ‘s’ command.

0:000> s -q 0 l?7fffffffffffffff 00000000033a0000

<snip>

000007fe`f21810a000000000`033a0000 00000000`00000001

The output of the ‘s’ command may be verbose.You will need to manually examine the addresses where ‘s’ finds hits. Most of these addresses will probably be in heap memory, we are looking for an address that matches a module.I snipped the above output to just show the relevant hit, an address inside of a loaded module.

The search of memory revealed that the heap 00000000033a0000 is used by the module useheap.dll, specifically it is part of the global ‘Blob’ class.

0:000> ln 000007fe`f21810a0

(000007fe`f21810a0)useheap!Blob::m_Heap

At this point we don’t know specifically what code in useheap.dll has allocated a lot of heap, however we have significantly narrowed the scope of the problem.We can now determine if there is a known issue with heap usage in useheap.dll that is addressed in a later version.We may also know from experience that this module uses a lot of memory under specific circumstances, such as a high volume of work sent to this service.

I hope this example helps the next time you have high memory usage and only have a user dump to troubleshoot with.Good luck!

Very often heap handles are stored in a global variables and got "heap" at the end of the variable name (like MS CRT's "_crtheap"). I usually try to locate such global variables by running "x *!_crtheap" or "x *!*_heap" or even "x *!*heap" or "x *!*heap*". There can be a lot of matches including function names but gloabl heap variables are usually visible. Of course this will not work if there are no symbols available.