JRun Threadpool settings

The other day I was looking at JRun's Threadpool implementation and it really took me SOME time to understand that piece of code. It is one of those codes which are not meant to be understood by others :D. It got me really confused about 'Active Handler Threads', 'min Handler Threads' and 'max Handler threads'. It was very much different from what I had assumed. Too much of 'creating runnables', 'swapping runnables', 'destroying runnables'.. phew.. I think this is the exact reason why Doug Lea and team had to introduce a standard implementation of ThreadPool in 'Tiger' release. Its so simple, neat and elegant I wonder why wasn't it introduced earlier in JDK.

Anyways, enough of cribbing. After my enlightenment of JRun's or CFMX's threadpool, I thought it would be nice to share it with you all. So what are these thread counts? (I am sure most of you would have it figured out. Neverthless.. :) )

Min handler threads - It is the number of web threads that will be spawned initially and will be waiting for HTTP requests. Which effectively means that it is the no of threads which will be waiting on serversocket.accept(). Thus it controls the no of requests that will be accepted concurrently. It is ideally the minimum concurrent users that you expect on the server. As soon as a thread gets a client request (i.e comes out of serversocket.accept()), it enters into a throttle before processing the request. This is where active handler threads come into the picture. Before the thread starts processing the request, it spawns another thread, if required, which can listen to the incoming requests.

Active handler threads - This decides how many requests would be concurrently processed. The throttle we talked about above, allows a maximum of "active handler" threads to continue and rest of threads wait until another thread exits the throttle. This along with "min handler thread" controls the throughput of the server. The value of active handler threads must be between min Handler count and maxHandler Count.

Max handler threads - This is maximum number of threads that can be created in the pool. This includes the threads queued in the throttle + threads processing the requests + threads waiting on the server socket. Once the server reaches the max Handler thread count, server will start denying the request throwing "Server Busy Error". You can see the "Server Busy" error even without server reaching the "max handler" limit if the thread in the throttle queue timeout. So if you see this error, dont start increasing the max handler thread count. You might need to tune all the three counts.

Having both MinHandler and activeHandler counts help JRun in addressing any sudden spike in the load. Lets say your minHandler count is 20 and activeHandler count is 40 and suddenly you have 40 concurrent requests, all of them will be served without any queuing and delay. When the load eases down on the server, it will let the extra threads die and bring the thread count down to minhandler count i.e 20.

The next question that naturally comes to the mind is what should be the appropriate values of these for my server? Having too less value for it would mean that you are not utilizing the potential of the server well and requests start queuing up even though server can handle it. Having a too high value for these would mean too many context switches and the server performance will deteriorate. (Too many context switches means CPU is busy scheduling the threads rather than executing them and thus hurts the performance) So what should be the appropriate value? well.. there can not be one answer or a formula to compute these values. It depends on your application, the traffic that your application expects, memory and processors of the machine on which you are going to run it etc etc.

By default the values in "ColdFusion standalone" are

- Min Handler Thread - 1
- Active Handler Thread - 8
- max Handler thread - 1000

min and active counts here are fine for a development machine but definitely not for a production machine. And in my opinion the value of 'max handler' is bit high even for a production machine. Creating a large no of threads does not necessarily increases the throughput of your server. It can actually lower it down because of the high no of context switches VM will have to make. Moreover, it might not be possible to create 1000's of threads because of OS limitations. On many of the OS, you will get an OutOfMemory Error because the VM will not be able to spawn so many native threads for you. I think max handler count in the range of 300-400 should be good enough.

Regarding tuning these counts, there are huge no of articles around which will tell you how to go about it. Since notion of these counts exist on all the application/web servers, articles need not be CF or JRun specific. In brief, you would need to run some kind of load tests with different values of minHandler and active handler counts, note the throughput and plot a graph. This graph should help you arrive at the appropriate value for these settings.