Pages

Thursday, April 11, 2013

This applies to vSphere vCenter Server version 5.1.x but may pertain to other versions.

I didn't find much info out there when Goggling so I'm adding what I found here. I searched for:vCenter server high memory utilizationVMware vCenter java high memoryVirtualCenter Server out of memoryVMWare vCenter Inventory Service memory

We upgraded our vCenter server to 5.1.0-947939, after which the server (with 4GB of RAM) was starving for memory. There were 5 instances of JAVA.EXE, each eating up a lot of RAM. The worst single instance of JAVA was using 1.5GB of RAM, the next one was nearly at 1GB.

In my searches I eventually found someone that was able to tweak the Java Heap Size default. There are many vCenter Server services that use Tomcat and Java, when you add them all up... ouch, it consumes all the RAM and brings the server to a grind.

What I found:
The main culprit was the "VMWare vCenter Inventory Service." This had the 1.5GB Java process. I found you could actually shut it down and the vSphere Client would still work. I won't pretend to know exactly what this service is for; but it seems to store the client INVENTORY data in a memory cache. I have only about 30 VMs and 3 hosts, hardly needing 1.5GB of cached info I would guess.

In addition to the Inventory Service, I found you could shut down the "VMware VirtualCenter Managment Webservices" also; however, this will cause an IE type error message in the vSphere Client if you try to look at OVERVIEW on the PERFORMANCE tab.

If you don't want to use the web-client, you can shut down "VMware vSphere Web Client" for a big chunk of memory also. Then if you do that, you also don't need "VMwareVCMSDS" which provides LDAP services (but it doesn't use much memory.) If you want to use the Web Client, you do need the LDAP service running though.

The other services are as such:

VMware Log Browser: not needed if you're not using the web client. Not a huge hog though.

VMware USB Arbitration Service: not a huge hog.

VMware vCenter Orchestrator Configuration: I don't use this, I have it disabled so I don't know what memory impact it has.

VMware VirtualCenter Server: The main service - you need this!

vCenter Single Sign On - does eat up some memory, but is nice to have.

VMware vSphere Profile-Driven Storage Service - I'm not really sure what this does to be honest. I turned it off to no noticable issues.

THE FIX

If you want the vCenter Inventory Service, and the Web Client running, there is a way to trim down the memory that JAVA uses. You can tweak config files (wrapper.conf) on the vCenter Server. Again, the biggest hog is the Inventory Service. Here is how to reduce the memory requirements.

Go to: C:\Program Files\VMware\Infrastructure\Inventory Service\conf

Save a backup copy of WRAPPER.CONF

Edit WRAPPER.CONF with a text editor

Look for the section: # Maximum Java Heap Size (in MB)

Modify "wrapper.java.maxmemory" Mine was set to 3072, I changed mine to 384.

Save the file.

Open Task Manager if you wish, and look for the JAVA process using the most RAM.

Shutdown / Restart the service: VMware vCenter Inventory Service.

Observe the memory difference.

It didn't comply 100% to the setting for me, but it did keep it to around 500MB in my case.

I kept all the services running and just tweaked the wrapper.conf files. My server went from 3.95GB of ram used (and running like crap) to 2.8GB used and running pretty good.

If you don't have many hosts and VMs you could have it running lean and mean (under 1GB of RAM) by just running the services below. This may be at the expense of vSphere Client performance and not seeing overview performance charts.

How is your vSphere Client performing? We found it so slow that we actually increased the Java heap size. Our vCenter VM is now 32GB RAM, with the biggest java.exe process regularly consuming 12GB+. Lowering the heap size is great for reducing RAM usage on the server itself, but will often come at the expense of client application performance.

One to watch out for is enabling VM Flash Cache in 5.5 U1. This can exhibit the same symptoms, high memory and CPU use by the JAVA SE processes but changing the memory limits does nothing to help. Link enclosed.

Some times i just think what will be some customers life without help like this one found on internet.The softwares producer have take more carefull about their products and "new features".I read a lot of tech material from vmware before upgrade and after, i have lost about 8 hours trying to figure out my vcenter java memory problem, than i found your site.I use some of your sugestions and like magic my memory that was 97% (3,97 gb) now is now 54%.