I have built a custom message server in Java that takes a stream of messages and delivers each message to its client (1:1, drop msg if not connected - very simple). I am running Tomcat 7 on Win7x64 & Java 7 and am using the NIO connector (implemented a Comet servlet).
It works great but I am now looking into scaling that beast and am currently seeing about 85kb of RAM allocated for each connected client. 10.000 clients @ under 900MB and scaling linearly. (I am not doing anything else but holding the connection yet) That's quite a lot to my opinion, so I am wondering whether there are some tweaks to make Tomcat or Java save more memory with their NIO impl. All the Tomcat settings I tried so far did not affect this at all.

Does anybody have experience how to put Java or Tomcat on a memory diet regarding socket connections?

UPDATE:
I am now down under 70kb/connection by trimming the socket buffers and some other tomcat internals. Not sure how this now influences throughput. I've also tried it on 32bit / 64bit linux with the same result.

I did a small load test and connected 10.000 clients. In the server code I did after every 100 clients a System.gc() and then printed the Runtime.totalMemory()-Runtime.freeMemory() to the console.
–
DanielApr 13 '12 at 17:29

Hi. Can you please share your best Tomcat / JVM settings? Also, did you use the NIO connector? 10x
–
JRunMay 9 '14 at 11:30

1 Answer
1

After quite some research and playing around I got to the conclusion that it is simply not possible with Tomcat to handle a huge amount of concurrent connections with reasonable amount of memory. (I'd still be happy to be proven wrong here btw)

It's a Java IO framework that builds on Java's new NIO architecture and seems very well designed and written. You can stack some lightweight modules together and create a mini webserver or simply handle the TCP connections yourself in an asynchronous way.

I ran a loadtest on EC2 and made it to a mindblowing 7 MILLION connections @ only 1.5GB of RAM! (like with the tomcat test I did nothing but store the connections, so a real app will of course consume a bit more mem, but 200 bytes / connection "overhead" is nothing!) And it only stopped there because I limited the Java VM to 1.5GB, I am sure a C10M test would be easily doable.