Fog Creek Software
Discussion Board




Java multithreading limitations

Sorry if this is a bad place to post this, but no one I've spoken with seems to have a clue.

My problem is that I am trying to write a normal Unix-style multithreaded TCP server (thread per connection model). I can handle no more than 916 connections, because any more than that (917+), and the VM will crap its pants. If I use multiple VMs on one machine, it works fine.

This is using J2SE 1.4.1, on a Linux box.

Do I need to be using J2EE? Or is the thread-per-connection model obsolete? I considered just multiplexing connections myself, but would rather not do so, if there's a better way. Is there a new server-design paradigm that is not mentioned anywhere? Anyone know of any books or web sites that would discuss other ways of implementing servers that won't hammer the VM so badly?

Thanks a ton!

Frustrated Java Programmer
Friday, January 24, 2003

Dude, as far as I know the "thread per connection" model is a problem as soon as you have large numbers of connections and you want decent performance. I'd suggest a google search for connection pooling.

Good luck! This is probably going to involve some JNI on your part...

Astarte
Friday, January 24, 2003

My thinking was simple: Many FTP servers use it, and some (i.e. cdrom.com) handle large numbers of connections on it.

I'll probably wind up using connection pooling, but I was just trying to figure out how the VM could fail, when it doesn't hammer my system: My system does not noticably slow down with that many threads (or twice that many, split among 2 VMs), and I still have free memory.

Just seems wierd and arbitrary.

Frustrated Java Programmer
Friday, January 24, 2003

Try the option
-Xno_crap_pants_when_more_than_916_threads.

(916?  But it's not even a power of two!)

Alyosha`
Friday, January 24, 2003

Check out the new java.nio package, available since J2SE 1.4.0.

/Daniel

Daniel
Friday, January 24, 2003

Good chances are J2EE will make no difference.

Even more within an J2EE app server (EJB crap) you cannot have a listener socket (ServerSocket) and you have no control over the threading whatsover.

Anyways there's a good chance you're running out of resources - and that makes the JVM crap out. Check your design and find one that is less resource intensive or that can limit the resource usage on the system.

Dino
Friday, January 24, 2003

Since you say that Multiple VMs on the same machine are OK, I think that it is the VM that is running out of resources.

Which VM are you using, Jikes?  Check the documentation to see what restrictions there are on a single VM.  Is there a limit to the memory available?  Is there a buffer than can be made larger?

Is it possible that you can put together a few demo classes to try out on other machines, to see what happens on other machines?

Ged Byrne
Friday, January 24, 2003

I agree with Daniel.  Check out java.nio.*.  I had this same problem years ago and wish I had this.  Project over now....

Howard
Friday, January 24, 2003

You do not say what platform this is on. If it is unix you may be running up against some per user or login file handle limit. If I recall, the command is setrlimit to check and set this.

DB
Friday, January 24, 2003

Replying to my own message. I did not see the Linux box mentioned the first time.  Do check setrlimit.

DB
Friday, January 24, 2003

First, check out the Java async IO stuff in 1.4. This would be the recommended way to go. Also, if you have time, try adapting your code to work within a event framework such as SEDA (http://www.cs.berkeley.edu/~mdw/proj/seda/). The best performing web server, Zeus, uses a similar paradigm.

If you don't have time to port your connection code to use async IO, then try using a new VM such as BEA Jrockit (formerly Appeal). It maps multiple Java threads to a single OS thread to improve scalability.

It was tested to be able to go to 6000+ concurrent connections: http://www.volano.com/report/index.html

Also, make sure you increase your file/process limits on Linux.

James Park
Friday, January 24, 2003

Instead of porting your code to java.nio, you probably should see if tuning Linux or the VM or switching VMs makes a difference.  IBM's VM is supposed to get good performance on Linux, and there is JRockit, gcj, and others as well. 

Check out the Volano benchmark at http://www.volano.com/report/index.html for ideas for tuning Linux -- you might need to raise the max number of file descriptors a process can have open.

Colin Evans
Friday, January 24, 2003

Sounds like the process is hitting an 1024 descriptor limit. Run 'limit' from the command line; if it says:

'files: 1024', or 'files:1000' or something close enough to 916, this is possibly the problem - 'limit files 4096' should solve that. I've never noticed if there's a 'limit threads' or 'limit processes', but if there is, that's another limit you might be hitting.

Another possible explanation (speculation ...) is that the JVM is using select() and has FD_SETSIZE set too low (e.g., 1024). To solve that you would need to recompile the JVM with a higher FD_SETSIZE, or switch to a JVM that uses poll() or another mechanism.

Ori Berger
Sunday, January 26, 2003

http://www.kegel.com/c10k.html

Dan Maas
Sunday, January 26, 2003

*  Recent Topics

*  Fog Creek Home