John,
I have made a few experiments with your test server on my FreeBSD box using ccl-1.5. I had to adjust the backlog parameter in usocket:socket-listen so that could cope with higher concurrency values, but other than that, I could see nothing abnormal.
Before going to the CCL mailing list with your problem, consider upgrading CCL to a more recent version.
-Hans
On Thu, Aug 19, 2010 at 12:27, JTK jetmonk@gmail.com wrote:
On Aug 18, 2010, at 10:11 PM, Hans Hübner wrote:
John,
what you are seeing is probably related to the fact that the default multi threaded task master of Hunchentoot is not clever about creating threads. It creates a new thread for every incoming connection, so it is easy to force it into situations where too many threads are created.
Personally, I am not using multi threaded Hunchentoot. Instead, I run it single threaded and behind a HTTP caching proxy (squid) that is configured to use if-modified-since headers to negotiate reloading stuff from the Hunchentoot backend. That strategy works well and copes with all load patterns that I have tried (and I have tried a few, using both ab and Tsung).
If you absolutely need threads, try the threading patch that has been submitted by Scott McKay a few weeks ago. It prevents Hunchentoot from creating threads without bounds, and might solve the problem for you. We will be incorporating that patch once it has been separated from the xcvb changes that won't go into the Hunchentoot mainline. Any reports on how it helps with artificial loads tests would be appreciated.
Here's a simple webserver that avoids H'toot altogether. It has a thread manager (make-safe-process) that avoids making too many threads at once. The threshold *max-procs* is very low and it never seems to hit the limit (because it prints warnings when it does).
If you run it as (do-dumb-server-loop :port NNNN) it runs great with just one request at a time, but fails with more than one thread (just like H'toot). Moreover, even in single-connection mode, the server performance deteriorates as one does more runs.
So it almost looks to me that CCL threads on the the mac might be broken somehow.
;; =========================================== ;; simple threaded webserver to exercise threads and sockets
(eval-when (load eval compile) (require 'bordeaux-threads) (require 'usocket))
(defparameter *lock* (bt:make-recursive-lock "proc-lock")) (defparameter *max-procs* 10) ;; max number of threads we allow
(let ((nprocs 0)) ;; counter for how many threads are running (defun safe-make-process (func) "A function that launches FUNC in a new thread only if there are not more than *MAX-PROCS* running; otherwise it sleeps for a bit" (bt:with-lock-held (*lock*) (format t "Making proc with ~A already existing~%" nprocs) (force-output)) ;; sleep until some processes die (loop while (bt:with-lock-held (*lock*) (> nprocs *max-procs*)) do (format t "Warning - nprocs=~A~%" nprocs) (force-output) (sleep 0.1)) ;; (bt:with-lock-held (*lock*) (incf nprocs)) (bt:make-thread (lambda () (unwind-protect ;; protect the decrement of the counter (funcall func) (bt:with-lock-held (*lock*) (decf nprocs)))))))
(defun do-dumb-server-loop (&key (port 9001)) (loop with listener = (usocket:socket-listen "127.0.0.1" port) for sock = (usocket:socket-accept listener) when sock do (safe-make-process (lambda () ;; I think we need to close over sock here or this thread's ;; loop might change what REPLY-FUNC sees (let ((this-sock sock)) (reply-func this-sock))) )))
(defparameter *response* "HTTP/1.1 200 OK Server: dumb Connection: Close Content-Type: text/plain
This is the content. ")
(defun reply-func (sock) ;; read headers and send content (let ((s (usocket:socket-stream sock))) (ignore-errors ;; eat client headers (loop for line = (read-line s nil nil) until (< (length line) 3)) ;; and send response (write-string *response* s)) ;; the socket close is protected by ignore-errors (usocket:socket-close sock)))
;; ===========================================
For the SBCL crash: This should be reported to the SBCL folks. I never had a lot of faith in SBCL's thread implementation and it also is possible that the locking patterns inside of Hunchentoot trigger the misbehavior, but we need to have detailed advise how to fix that, if we need to.
-Hans
On Thu, Aug 19, 2010 at 01:53, JTK jetmonk@gmail.com wrote:
Hello,
I'm finding that the speed of hunchentoot falls drastically for more than 1 simultaneous connection, at least for CCL on OS X. And more than a few connections just fails.
I downloaded the latest svn version from http://bknr.net/html/
To reduce any complicating factors, I loaded just h'toot and ran ccl on the shell command line, not in slime.
I tried a simple page:
CL-USER> (hunchentoot:start (make-instance 'hunchentoot:acceptor :port 4242)) #<ACCEPTOR (host *, port 4242)> CL-USER> (hunchentoot:define-easy-handler (say-yo :uri "/yo") () (setf (hunchentoot:content-type*) "text/plain") "Yo! This. Is. Content") SAY-YO
Then I tried the ab (apache bench) benchmark with c=1 (1 connection), and 500 iterations:
$ ab -n 500 -c 1 http://127.0.0.1:4242/yo
HTML transferred: 10500 bytes Requests per second: 138.26 [#/sec] (mean) <- ##### note speed ###### Time per request: 7.233 [ms] (mean) Time per request: 7.233 [ms] (mean, across all concurrent requests) Transfer rate: 22.82 [Kbytes/sec] received
Then I tried it with TWO concurrent connections:
$ ab -n 500 -c 2 http://127.0.0.1:4242/yo
HTML transferred: 10500 bytes Requests per second: 29.10 [#/sec] (mean) <- ##### note speed ###### Time per request: 68.722 [ms] (mean) Time per request: 34.361 [ms] (mean, across all concurrent requests) Transfer rate: 4.80 [Kbytes/sec] received
It's nearly 5x slower with just 2 simultaneous connections, going from 138 to 34 requests/sec.
10 connections gives the same speed as 2, and 30 fails because connection was reset by peer (maybe my fault for not having enough open files available, but it should never open more than 30 filehandles at once, no?).
I'm running on a Mac OS X 10.6 with CCL "Version 1.4-r13119 (DarwinX8664)"
Can anyone else reproduce this? Is it some threading problem?
John
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel