The Old Joel on Software Forum: Part 4 (of 5) - Threads Supported on a Server

Threads Supported on a Server

This may be a dumb question.

Looking at a server's specification - is it possible to determine the number of concurrent threads it can support?

This will be for a Java application.

Ram Dass
Friday, February 13, 2004

In general terms, I think that it would be determined by how much of each type of resource is being used by each thread.

Memory, CPU cycles (time) per operation and/or disk/network usage based on the application.

Just curious, is this for some kind of net server that handles connections? Are you doing a thread per connection or using a pool?

Wayne
Friday, February 13, 2004

I think that without running tests you could not just look at a server's specs and know how your application is going to perform.

At the least you should fire up the Windows Perfmon and run your app on a few different machines.

Wayne
Friday, February 13, 2004

There should be as many threads as there are CPUs. :)

Brad Wilson (dotnetguy.techieswithcats.com)
Friday, February 13, 2004

> There should be as many threads as there are CPUs. :)

I know the theory behind this statement and without wanting to divert the thread too far ... how do the new hyperthreaded CPU's affect this guideline?

Should it simply be extended to a thread per logical CPU (ie. trust the chip maker to be able to keep the functional units on the chip busy) or left as a per physical CPU limit.

My reaction would be to go with logical CPU but I'm interested in other opinions.

Rob Walker
Saturday, February 14, 2004

UNIX? Linux? Be? Win95? W2K3? I bet answers would be slightly different.
I would not try to push Win95 hard for example :)

WildTiger
Saturday, February 14, 2004

Rob, Brad wasn't bein serious, he was trying to make a point. I'd guess that windows and many other applications use multiple threads. You think IIS doesn't use threads? Or other web/application servers?
Also, from what I understand, your operating system actually recognizes "hyperthreaded" cpus as multiple CPUs, so I don't think its a good idea to try and circumvent what the OS thinks.

Ram-
I believe what wayne was getting at, is that you shouldn't even be *asking* this question, because threads aren't resources that should just be spun off arbitrarily. People pool threads for a reason. Each additional thread causes a performance hit.

vince
Saturday, February 14, 2004

Ever since Java 1.4, Sun has provided the backend support for a specific function call that determines the number of CPUs exposed by the system. The call is always there in Java, but only some OSes+Platforms maps it to a real function on the other side of the JNI river. I can't recall it off the top of my head, but I remember reading about it in Java Threading from O'Reilly and Associates.

Li-fan Chen
Saturday, February 14, 2004

Now I guess I misunderstood the question. Some of the more efficient multithreading models maps only as many threads are there are processors, and after that programmers are encouraged use non-blocking or other job distribution methods to parallize job scheduling and completion (see SEDA, cs.berkeley.edu/~mdw).. if you want ot find out how many green threads or os threads are supported by Java there are specific ways to find out. Partly from reading OS manuals (and forums online) and partly from using the java api (to do a load test).

Li-fan Chen
Saturday, February 14, 2004

Actually, I _was_ being (mostly) serious. IIS doesn't use nearly as many threads as most people assume it does. It uses a very small pool, and relies on overlapped I/O to make the most of those threads.

As for HT CPUs... I agree. I'd use the count of logical CPUs, not physical ones.

Brad Wilson (dotnetguy.techieswithcats.com)
Saturday, February 14, 2004

It is not a _dumb_ question -- there are no dumb questions. It is, however, a question that is symptomatic of Thread Happiness Disease, a peculiar disease which usually strikes programmers who are relatively new to large-scale multi-threaded software development.

Why's that? Because if you were writing an app where there was a reader thread and a writer thread, you wouldn't be asking the question.

No, if you're asking the question "_how many_ threads can I create?" then I can only assume that you must be thinking of writing some application that is going to create a _large_, _variable_ number of threads.

Don't _ever_ do that! If you're thinking of doing that, you've almost certainly got a bad design.

To actually answer the question, no, there's no way of telling that from the server spec because there's nowhere near enough information in the "static" facts about the hardware. Thread perf depends on "dynamic" facts like:

What other processes are going to be running?
What are those threads going to be doing?
How much memory do you have free?
How much contention, locking, busy waiting, context switching, blah blah blah, is there going to be?
What are the performance metrics that will determine when things have gotten too bad?

Now, you might be wondering "What about real-world server applications which do spin up a variable number of threads then? How do they decide how many is too many?"

It's quite hard. These things used to be configurable in IIS, but a few years back the IIS team had the realization that (a) you can't expect a human being to figure out what the ideal thread limit is, and (b) the ideal thread limit changes dynamically because of all the factors I mentioned above.

IIS therefore keeps a relatively small thread pool, and continually monitors its own performance, tweaking the thread pool count, thread priorities, etc, as conditions change. It tries to keep the server well-tuned dynamically.

This was non-trivial code to write. Though I've done quite a bit of optimization of software that runs in-process with IIS, the details of the IIS thread timing algorithms are way, way beyond my skills. I wouldn't recommend such an approach unless you've got a lot of experience in the field. (A huge performance lab with a wide range of server hardware and a dedicated staff of perf testers doesn't hurt either.)

Eric Lippert
Saturday, February 14, 2004

You are asking about threads in relation to network connections; it is also worth asking how many sockets can a server have open too.

Incidently, very high performance web servers (such as thttpd) use only one thread deliberately..

i like i
Sunday, February 15, 2004

Ouch! i like i was just caught in the 'can't see thread you're replying to' trap; Wayne mentioned network connections, and I kind of got carried away. Sorry.

i like i
Sunday, February 15, 2004