Saturday, September 2, 2017

Everyone in java development faces java.lang.OutOfMemoryError now and then, OutOfMemoryError in Java is one problem which is more due to system's limitation (memory) rather than due to programming mistakes in most cases though in certain cases you could have a memory leak which causing OutOfMemoryError. I have found that even though java.lang.OutOfMemoryError is quite common basic knowledge of its cause and solution is largely unknown among junior developers. Beginners books like Head First Java doesn't teach you much about how to deal with this kind of error. You need real experience dealing with production systems, handling a large number of user sessions to troubleshoot and fix performance issues like running out of memory.If you want to be good at troubleshooting and performance analysis, you need to learn some books on Java performance and profiling e.g. Java Performance The Definitive Guide By Scott Oaks or the Java Performance by Binu John. They are an excellent resource for senior Java developers and also teaches you tools and process to deal with an error like java.lang.OutOfMemoryError.

In this article, we will explore what is java.lang.OutOfMemoryError; Why OutOfMemoryError comes in Java application, different type of OutOfMemoryError and How to fix OutOfMemoryError in Java. This article is purely meant to provide basic knowledge of java.lang.OutMemoryError and won't discuss profiling in detail. For profiling read the books I have mentioned before.

What is java.lang.OutOfMemoryError in Java

OutOfMemoryError in Java is a subclass of java.lang.VirtualMachineError and JVM throws java.lang.OutOfMemoryError when it ran out of memory in the heap. OutOfMemoryError in Java can come anytime in heap mostly while you try to create an object and there is not enough space on the heap to allocate that object. Javadoc of OutOfMemoryError is not very informative about this, though.

If you are familiar with different generations on the heap and How garbage collection works in java and aware of new, old and permanent generation of heap space then you would have easily figured out this OutOfMemoryError in Java. Permanent generation of the heap is used to store String pool and various Metadata required by JVM related to Class, method and other java primitives.

Since in most of JVM default size of Perm Space is around "64MB" you can easily run out of memory if you have too many classes or a huge number of Strings in your project.

An important point to remember is that it doesn't depend on –Xmx value so no matter how big your total heap size you can run OutOfMemory in perm space. The good thing is you can specify the size of permanent generation using JVM options "-XX: PermSize" and "-XX: MaxPermSize" based on your project need.

One small thing to remember is that "=" is used to separate parameter and value while specifying the size of perm space in the heap while "=" is not required while setting maximum heap size in java, as shown in below example.

export JVM_ARGS="-Xmx1024m -XX:MaxPermSize=256m"

Another reason of "java.lang.OutOfMemoryError: PermGen" is memory leak through Classloaders and it’s very often surfaced in WebServer and application server like tomcat, WebSphere, glassfish or WebLogic.

In Application server different classloaders are used to load different web applications so that you can deploy and undeploy one application without affecting other application on the same server, but while undeploying if container somehow keeps reference of any class loaded by application class loader then that class and all other related class will not be garbage collected and can quickly fill the PermGen space if you deploy and undeploy your application many times. "java.lang.OutOfMemoryError: PermGen” has been observed many times in tomcat in our last project, but the solution of this problem are really tricky because first you need to know which class is causing a memory leak and then you need to fix that. Another reason of OutOfMemoryError in PermGen space is if any thread started by the application doesn't exit when you undeploy your application.

These are just some example of infamous classloader leaks, anybody who is writing code for loading and unloading classes has to be very careful to avoid this. You can also use visualgc for monitoring PermGen space, this tool will show the graph of PermGen space and you can see how and when Permanent space getting increased. I suggest using this tool before reaching to any conclusion.

Another rather unknown but interesting cause of "java.lang.OutOfMemoryError: PermGen" we found is introduction of JVM options "-Xnoclassgc". This option sometimes used to avoid loading and unloading of classes when there are no further live references of it just to avoid performance hit due to frequent loading and unloading, but using this option is J2EE environment can be very dangerous because many framework e.g. Struts, spring etc uses reflection to create classes and with frequent deployment and undeployment you can easily run out of space in PermGen if earlier references were not cleaned up. This instance also points out that sometimes bad JVM arguments or configuration can cause OutOfMemoryError in Java.

So the conclusion is to avoid using "-Xnoclassgc" in the J2EE environment especially with AppServer.

Tomcat to Solve OutOfMemoryError in PermGen Space

From tomcat > 6.0 onward tomcat provides memory leak detection feature which can detect many common memory leaks on web-app perspective e.g ThreadLocal memory leaks, JDBC driver registration, RMI targes, LogFactory and Thread spawned by web-apps. You can check complete details on htp://wiki.apache.org/tomcat/MemoryLeakProtection you can also detect memory leak by accessing manager application which comes with tomcat, in case you are experiencing memory leak on any java web-app its good idea to run it on tomcat.

How to solve java.lang.OutOfMemoryError: Java heap space

1) An easy way to solve OutOfMemoryError in java is to increase the maximum heap size by using JVM options "-Xmx512M", this will immediately solve your OutOfMemoryError. This is my preferred solution when I get OutOfMemoryError in Eclipse, Maven or ANT while building project because based upon size of project you can easily run out of Memory.here is an example of increasing maximum heap size of JVM, Also its better to keep -Xmx to -Xms ration either 1:1 or 1:1.5 if you are setting heap size in your java application

export JVM_ARGS="-Xms1024m -Xmx1024m"

2) The second way to resolve OutOfMemoryError in Java is rather hard and comes when you don't have much memory and even after increase maximum heap size you are still getting java.lang.OutOfMemoryError, in this case, you probably want to profile your application and look for any memory leak. You can use Eclipse Memory Analyzer to examine your heap dump or you can use any profiler like Netbeans or JProbe. This is tough solution and requires some time to analyze and find memory leaks.

How to solve java.lang.OutOfMemoryError: PermGen space

As explained in above paragraph this OutOfMemory error in java comes when Permanent generation of heap filled up. To fix this OutOfMemoryError in Java, you need to increase heap size of Perm space by using JVM option "-XX: MaxPermSize". You can also specify initial size of Perm space by using "-XX: PermSize" and keeping both initial and maximum Perm Space you can prevent some full garbage collection which may occur when Perm Space gets re-sized. Here is how you can specify initial and maximum Perm size in Java:

export JVM_ARGS="-XX:PermSize=64M -XX:MaxPermSize=256m"

Some time java.lang.OutOfMemoryError in Java gets tricky and on those cases profiling remains ultimate solution.Though you have the freedom to increase heap size in java, it’s recommended that to follow memory management practices while coding and setting null to any unused references.
That’s all from me on OutOfMemoryError in Java I will try to write more about finding the memory leak in java and using profiler in some other post. Please share what is your approach to solving java.lang.OutOfMemoryError in Java.

Important Note: From Tomcat > 6.0 onward tomcat provides memory leak detection feature which can detect many common memory leaks on Java application e.g ThreadLocal memory leaks, JDBC driver registration, RMI targes, LogFactory, and Thread spawned by web apps. You can check complete details on htp://wiki.apache.org/tomcat/MemoryLeakProtection. You can also detect memory leak by accessing manager application which comes with tomcat, in case you are experiencing memory leak on any java web app it's a good idea to run it on tomcat to find out the reason of OutOfMemoryError in PermGen space.

Tools to investigate and fix OutOfMemoryError in Java

Java.lang.OutOfMemoryError is a kind of error which needs a lot of investigation to find out the root cause of the problem, which object is taking memory, how much memory it is taking or finding dreaded memory leak and you can't do this without having knowledge of available tools in java space. Here I am listing out some free tools which can be used to analyze heap and will help you to find culprit of OutOfMemoryError

1) Visualgc

Visualgc stands for Visual Garbage Collection Monitoring Tool and you can attach it to your instrumented hotspot JVM. The main strength of visualgc is that it displays all key data graphically including class loader, garbage collection, and JVM compiler performance data.

The target JVM is identified by its virtual machine identifier also called as vmid. You can read more about visualgc and vmid options here.

2) Jmap

Jmap is a command line utility comes with JDK6 and allows you to take a memory dump of the heap in a file. It’s easy to use as shown below:

jmap -dump:format=b,file=heapdump 6054

Here file specifies the name of memory dump file which is "heap dump" and 6054 is PID of your Java progress. You can find the PDI by using "ps -ef” or windows task manager or by using the tool called "jps"(Java Virtual Machine Process Status Tool).

3) Jhat

Jhat was earlier known as hat (heap analyzer tool) but it is now part of JDK6. You can use jhat to analyze heap dump file created by using "jmap". Jhat is also a command line utility and you can run it from cmd window as shown below:

jhat -J-Xmx256m heapdump

Here it will analyze memory dump contained in file "heapdump". When you start jhat it will read this heap dump file and then start listening on HTTP port, just point your browser into port where jhat is listening by default 7000 and then you can start analyzing objects present in heap dump.

4) Eclipse memory analyzer

Eclipse memory analyzer (MAT) is a tool from eclipse foundation to analyze java heap dump. It helps to find classloader leaks and memory leaks and helps to minimize memory consumption.you can use MAT to analyze heap dump carrying millions of object and it also helps you to extract suspect of memory leak. See here for more information.

5) Books to learn Profiling

As your Java experience grows, expectation also grows in terms of niche skills like analyzing performance issue and comfortable with profiling. You won't normally learn those skill unless you take extra effort. In order to effectively deal with the error like java.lang.OutOfMemoryError, you should read good books on troubleshooting and performance tuning e.g. Java Performance The Definitive Guide By Scott Oaks as shown below:

you should also include "java.lang.OutOfMemoryError: GC overhead limit exceeded" , this error comes when Garbage Collector spends excessive amount of time doing GC and recovers a little, say 2% of heap. possible reason for this OutOfMemoryError could be that you are creating lots of temporary object inside loop or recursive function. to fix this review your code and fix the cause or else you can try by increasing maximum heap size using -Xmx. you can also disable this alert by following JVM options : -XX:-UseGCOverheadLimit

very nice post. our application was also getting stuck due to OutOfMemoryError due to use of large number of String. I solved it by minimizing number of String object by StringBuffer. Now this setting of Perm generation will also be helpful to make application more efficient.Thanks a lot.Sanjeevall-technicals.blogspot.com

Hi Ramamurthy, from your description it looks like a classloader leak to me. try deploy -undeploy operation few times and if it appears it means it sure short classloader leak. now to fix that you need to find what is causing this leak, which may be tricky, you can use the tools I have listed or you can also try asking on javaranch or tomcat forums.

Rama are you using JDBC driver, I was suffering from same problem but thanks to spring it warned me.SEVERE: The web application [/OutOfMemoryTest] registered the JDBC driver [oracle.jdbc.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

Another way to solve "java.lang.OutOfMemoryError:PermGen Space " in tomcat due to JDBC driver is moving jdbc drivers jar to tomcat lib and let common classloader load the driver and not the webapp loader.

Hi, I am getting java.lang.OutOfmemoryError java heap space while reading data from database and manipulating it on my program. I load a large resultset which contains 1000 rows and it seems Ojbects are getting larger than heap. please help.

hi Anonymous java.lang.OutOfMemoryError: unable to create new native thread Exception comes when a request to create a new thread can not be full filled due to no memory. try increase PermGen Space by using command export JVM_ARGS="-XX:PermSize=64M -XX:MaxPermSize=256m" and let us know how it goes.

What is difference between OutOfMemoryError in PermGen and OutOfMemoryError in Java Heap , isn't it Perm Gen is part of Java Heap and if Heap runs outofmemory than how can Perm Gen space remain empty, please help?

@Anonymous As I explained them in this OutOFMemoryError tutorial they are entirely different. yes Perm Gen space is part of Java heap but its a separate generation and space is not adjusted with other region in heap. also different parameters are used to define size of PermGen space and Heap space in Java.

I was getting ava.lang.OutOfMemoryError: PermGen space in Eclipse 3.7 and adding -XX:MaxPermSize=512M in eclipse.ini file solves that for me.Thanks for clear explanation of OutOfMemoryError in PermGen Space.

@amidst, Can you please explain where exactly you are getting java.lang.OutOfMemoryError, would be great if you post exact error, as there are different types of OutOFMemroyError in Java and different ways to handle them.

Hi Javin, I am getting out of memory error still memory is available(please refer below logs). Another thing is GC can not recovered at least 2% of memory so getting also "OutOfMemoryError: GC overhead limit exceeded". Can you please suggest how to solve it.

@Bhavesh , possible cause of java.lang.OutOfMemoryError: GC overhead limit exceeded is that it comes if Full GC takes longer time than usual and amount of memory available in both new generation and old generation is smaller than there specified limit. -XX:+UseGCOverheadLimit JVM option in Java 6 specify the limit the proportion of the VM's time that is spent in GC before an OutOfMemory error is thrown. play with this option if it help or try to find out cause of long GC duration.

helloI really didn't understand the text very will it is clear text but for beginner like me It is confused somehowwhere I have to put this sentence "-XX:PermSize"? I'm using jgrasp in programming but not for web applicationsthe error was:Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"I read som other subjects they said to use run arguments then text box will appears then write for example "-Xmx1024m" but nothing happened!if you can give me a hint.thanks in advance (:.

Hi Gina, I don't know about jgrasp. are you using any IDE like Eclipse or Netbeans? anyway on very basic level, every Java program starts with "java" command in both Windows and Unix. -XX:PermSize is a command line argument to "java" command.

Hi Gina, If you are using Netbeans,then Select Project in Project explorer (I hope its Java Project) and right click->Properties. Then look for run time settings, provide your Java memory settings there.

Memory leak in PermGen space is a serious problem and require lot for attention, investigation and troubleshooting as author says "Its hard part" but simply increasing PermGen space is not a solution you need to identity the cause first before applying any solution. blog post on Oracle https://blogs.oracle.com/fkieviet/entry/classloader_leaks_the_dreaded_java has great discussion on finding and troubleshooting classloader leaks which is primary cause of running OutOfMemory in PermGen .

Hello Sir, We are getting java.lang.outofmemoryerror permgen space in Weblogic Server. we are using following JVM parameters:-Xms2G -Xmx2G -XX:PermSize=512m -XX:MaxPermSize=512m -XX:+UseParallelGC -Xnoclassgc. I am suspecting memory leak issue because we are using ThreadLocal variable or as you may mentioned may be ClassLoader Leak as we are also using JDBC API but I have no clue how to check PermGen space and which objects are stored in PermGen memory. any help

@Kavita, try to remove -Xnoclassgc parameters as it prevent classes from PermGen not to be garbage collected which results in java.lang.OutOfMemoryError: PermGen space. I guess -Xnoclassgc also one of the reason of "java.lang.OutOfMemoryError: GC overhead limit exceeded" which I am currently investigating.

Hello, We are getting "java.lang.OutOfMemoryError: thread creation failed" in Eclipse IDE while working on Android application. I am not sure whether OutOfMemory is related to Eclipse or Android, please advise.

Nice article.... I think OutOfMemoryError is a Fatal Error in VM and from Java 6 onwards Virtual machine produce complete stack trace of OutOfMemoryError. You can also use JVM parameter -XX:OnOutOfMemoryError= to run any command in an event of first OutOfMemoryError thrown by JVM. You can also generate heap dump on OutOfMemoryError by using -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath= to specify directory where heap dump needs to be created. With this much information its easy to diagnose any memory leak in Java, but as you said most of the time just increasing size of heap or permgen memory area.

One tip to solve OutOfMemoryError in heap space and permgen is to look output of full garbage collection and find out what triggering it whether its triggered by full permgen space and or or full old generation space in heap. that will give clue for sure.

Hi, Most Out of memory errors are from memory leaks introduced by java developers in the java application code/programs, as the default heap size is enough to handle almost all possible application scenarios,you can get prompt response and possible solutions for all your java out of memory errors at http://loasoftwares.com/servicesLink.action#loaTechnicalService

If you still face the problem after following the above methods, like i have faced after performing given below step and also tried to increase the values from 18 to 512 and from 512 to 1024"-vmargs -XX:PermSize=128M -XX:MaxPermSize=512M "

but still i was facing problem then i had deleted the Build files under bld folder and increase the the size upto 512M. It solves my problem.

There are lot more about java.lang.OutOfMemoryError than java.lang.OutOfMemoryError: Java heap space and java.lang.OutOfMemoryError: PermGen space. First of all Java 8 has removed PermGen space and introduced Meta space, so be ready for a new kind of OUTOFMEMORYERROR in Java 8. Apart from these big two there are couple of more kind of this, e.g. :java.lang.OutOfMemoryError: GC overhead limit exceededjava.lang.OutOfMemoryError: unable to create new native thread java.lang.OutOfMemoryError: Requested array size exceeds VM limit java.lang.OutOfMemoryError: request bytes for . Out of swap space? java.lang.OutOfMemoryError: (Native method)

which all comes with different reasons e.g. GC overhead limit exceeded comes when Garbage Collector spends more time collecting it but not able to collect enough.

Hello there, We are getting java.lang.OutOfMemoryError OUT OF SWAP SPACE in our JBoss server, exact error is this :

Exception in thread "main" java.lang.OutOfMemoryError: requested 8679869200bytes for Chunk ::new. Out of swap space? and our JVM crashes. I have tried increasing both Java Heap memory and PermGen Space memory by -Xmx and -XX:MaxPermSize, upto 3GB, Since we are running on Solaris 32-bit Machine and our JVM is also 32-bit JVM with JRE version Java 6 update 20. Any pointer would be helpful.

I am getting java.lang.OutOfMemoryError: Map failed, while creating a memory mapped file of 2GB. This exception is thrown when FileChannel trying to call map() method. Since I am running on 64-bit machine and have enough RAM almost 6GB to support this Memory mapped file, I am wondering why Java throwing java.lang.OutOfMemoryError: Map failed.

Please let me know how to fix this error, any JVM option to solve this?

I have a question/scenario that i want to ask/share with you.I am facing following issue on production server:There are two production server not virtually connected totally different machine having different database and different IP, but both are working under one load balancer. OS is Windows server 2003. RAM 16GB.JBoss configuration: 5.1, heap size 1024mb.We have a servlet (Web Container we are using is JBoss) which takes request coming from mobile phones, process it (which is lengthy process) then communicate with our server (MEAP server, built using C and C++) using socket. This MEAP server executes lots of queries on database (Oracle) also call webservices if necessary and respond to the servlet, then servlet makes a proper response and give it back to the mobile phone. But when production server is running under lots of load (25000 requests per day), JBoss is getting down with "java.lang.OutOfMemoryError unable to create new native thread".What i have done :1. Check system log which is clean when this issue occure.2. Check db connections. (But we are not executing queries from servlet).3. Check task manager java process takes more than 700mb i dont know why but there are two java processes and both are taking more than 500mb each. I dont know why but our MEAP server which is handling db connections and webservice calls having lots of loads, takes much less memory than java process.4. I have taken heap dump and also install eclipse memory analyzer but dont know how to check memory leaks if any are there, so please guide me with that.

I am fail to find root cause i also don't know much about tools, i don't know whether the problem is in servlet or in server environment or in concurrency.If you want additional information i would like to share with you.Please help me if you have time.

Hi Dheeraj, Operating systems have limits on the number of threads that you can create, I think your application/JBoss is violating that limit. Check how many threads are running when this error comes, you can use command ps -eLF| grep -c java and try to configure maximum number of thread, may be JBoss might have config, you need to search for that. If anyone has faced this issue previously or know how to configure JBoss to handle such load, please let us know. I will also post if I come across exact reason or configuration change required to solve this error in your scenario.

i have created a Jasper Server Report and deployed into Jasper server, i used a property "net.sf.jasperreports.page.break.no.pagination:apply" to get the data into two different sheets, Report runs find in Jasper Server, but when i try to export that report into Excel i am getting this "ava.lang.OutOfMemoryError: Java heap space" exception, i checked Java heap space is already 1024MB and Max size is 2048 MB, any solutions for this?my report have more that 3500 Records in tabular format..

@Anonymous, 3500 records is quite less for 1GB heap memory unless your individual object is really huge. I think you check if you really setting that much heap? Print the memory stats using freeMemory() and totalMemory() method of Runtime class. On different note, post your query to jasper report forum, may be a setting or two missing.