Where did that large object come from?

Sometimes, when you're running a Java application, you'll have a
problem where your application is allocating a large object which you
weren't expecting it to. In this article I'll talk about ways that you
might notice this is happening, and how you can use different features
of IBM's JVM to work out where in your application is causing the
problem.

Pre-reqs

I'm using an IBM Java install - Java 5 SR10 or above and Java 6 SR5 or above will work.
If you're using version 1.4.2 of IBM Java then try using the the ALLOCATION_THRESHOLD approach explained in this Websphere technote.

Spotting the symptoms

How might you notice that your application is allocating unexpected large objects?

In IBM support, we've found that of the most common ways customers
are alerted is through the output from -verbose:gc. Enabling
-verbose:gc causes the JVM to write a log of the garbage collector's
activity to the console. You can learn more about -verbose:gc output by
reading the relevant "Garbage Collector diagnostics" section in the IBM Java Diagnostics Guide, and a useful tool for analysing the output from -verbose:gc is the Garbage Collection and Memory Visualizer.

-verbose:gc is an XML log, which contains a section for each garbage
collection the JVM does. One of the pieces of information included in
this section is the size of the object which triggered the garbage
collection. Here's an example section from a -verbose:gc log.

The second line of the section shows you the size of the object which triggered this garbage collection:

<minimum requested_bytes="5242880" />

A 5Mb object! Who on earth allocated that?

Finding the causes

One thing you could do to try and work out where the object was
allocated is to take a heapdump, then examine its contents using a tool
like the Eclipse Memory Analyzer. However, there is a problem with this approach.

A heapdump will only show you a snapshot of the state of the Java
heap. It will show you which objects refer to your large object, but
that might not be enough information to let you work out where the
object came from. For example, what happens if you find out that your
large object is an element in a Hashtable? There may well be lots of
different places in your application which call Hashtable.put() and you
will have no way of knowing which one added the object you are
interested in.

Fixing the problem

Luckily, the JVM has a facility designed to allow you to catch your
application "in the act". By setting a command-line option you can take
advantage of the JVM's built-in capabilities for taking dumps. Here's
an example of the option:

You can see that JVM has output a JVMDUMP006I line which shows that
it has detected an object allocation of a size within your specified
range. Helpfully it's told you what class the object is (here, it's a
byte array - see the JNI documentation
for details of these signatures). After that there are some details
about the Java thread, including its name, and then there is a full
Java stack trace, showing you where the application was. The stack
trace includes file and line number information - useful for getting to
the bottom of the problem quickly.

You can also use this option to home in on a particular size which
is causing you problems by specifying a range of sizes you're
interested in. For example:

-Xdump:stack:events=allocation,filter=#5m..6m

will cause a Java stack trace to be triggered when an object between 5Mb and 6Mb is allocated.

I hope you've found this post useful, and I'll be happy to answer
any questions you have about tracking down large object problems in the
comments.