On 16 Nov 2008, at 19:08, Hans Hübner wrote:
Hi Cyrus,
when you see worker threads accumulate, do you also see that there is a large number of connections to Hunchentoot that are not being closed? Or are there just threads, but no connections? Are you running Hunchentoot behind a proxy/frontend, or standalone? How many dead workers did you see?
NB: I do certainly think that it is possible to write threaded web servers that work, I just believe that it is hard, in many respects. Threads, in my personal opinion, are not the best hammer to solve the I/O multiplexing problem that needs to be solved in a web server. Certainly, the unbounded worker thread creation strategy that Hunchentoot uses is not suitable for servers that see load peaks, which is why I recommend not using threaded Hunchentoot for such sites.
That said: I do have stability issues with my non-threaded Hunchentoot installation on FreeBSD, too. I use a multi threaded SBCL, but run Hunchentoot in a single thread behind squid. In some situations, no new connections are being accepted for no apparent reasons, but I failed to properly analyze the problem last time it happened as the customer was already nervous. I have seen this happen two times in the last four months, on a moderately busy site. Thus, the problem may actually not be related to Hunchentoot's threads usage (I'm running non-threaded, you run threaded), but could as well be located in SBCL's thread implementation of FreeBSD, in usocket or somewhere else.
Any further data points would help getting down to the bottom of this.
As for future work on Hunchentoot: We do have the new connection manager class in place which is meant to support the implementation of thread pools. Thread pools would help putting limits on the number of threads created, helping with getting through load peaks. I do not personally need such a connection manager, but rather want to spend some time on making Hunchentoot be able to use single threaded I/O multiplexing using select/kpoll/whatever.
-Hans
On this, I've had no trouble with multi-threaded Hunchentoot for a moderate traffic site (~10 requests per sec, generally a few threads working at a time), on Linux. Rob