April 2011 - tbnl-devel@lists.common-lisp.net

Re: [hunchentoot-devel] Trouble with finish-output in process-connection
by Hans Hübner 04 Apr '11

04 Apr '11

Hi Ilya, please Cc the list in all emails concerning Hunchentoot. Reply below. On Mon, Apr 4, 2011 at 10:14 AM, Ilya Perminov <iperminov(a)dwavesys.com> wrote: > Hi Hans, > > Why is CLOSE always called with :ABORT T? It seems to me it is only > necessary when an IO error occurs and the connection have to be dropped. > If it is possible to change the code to use CLOSE without :ABORT T in > normal cases, it will be completely safe to use FORCE-OUTPUT inside of > the loop. I am not sure the change will resolve the "hanging in > FINISH-OUTPUT" problem, but it will make debugging easier. Ah, now I see what you mean. I will look at the code to find out why :ABORT is used later, but I suspect that there is no good reason to do that in non-error cases. > It would be really useful to find a simple way to reproduce the "hanging > in FINISH-OUTPUT" problem. If it is a SBCL bug, getting it fixed in SBCL > will be better than implementing a workaround. I agree with both points made. If there is a suitable workaround for older SBCL versions, it'd be even better. -Hans

1 0

[hunchentoot-devel] Trouble with finish-output in process-connection
by Andrey Moskvitin 04 Apr '11

04 Apr '11

Hi, A few days ago on the site lisper.ru I started using Hunchentoot from trunk. And found that the system stopped working in a few hours. I watched the behavior of the system during the day and found that some threads do not complete their work and hangs on "finish-ouptut" (in "process-connection"). When they accumulate enough, the work stops. I have this problem on Debian 64-bit with SBCL-1.0.47 and SBCL-1.0.37. I have not had this trouble when used Hunchentoot-1.1.1 After replacing the "finish-output" on "force-output" in the "process-connection" the problem was fixed for me. Andrey

3 7

[hunchentoot-devel] ssl crashing in hunchentoot. ssl patch works in CCL, but not SBCL on mac
by JTK 04 Apr '11

04 Apr '11

[cc'ed to hunchentoot list] I can confirm that ccl+hunchentoot on OS X 10.6 does not crash in ssl at a concurrency of 20 after 100,000 accesses (100% success in apache bench), whereas before it would crash after a few hundred. However, SBCL 1.0.47 is still freezing, but not coredumping. It just times out on the apachebench side, and in slime both the *inferior-lisp* and *slime-repl* stop responding, and the CPU is pegged at 100%. Using the toy server below, ab also causes an eventual freeze in sbcl but with CCL it mostly works, sometimes giving "SSL read failed - closing connection," which might be the fault of a premature close in the toy server (but I don't see how to do it otherwise, besides waiting until the client side closes). So this isn't a hunchentoot problem, most likely. Given that CCL+hunchentoot works perfectly maybe the problem is inside SBCL, or maybe just the Mac version of SBCL if other platforms now work. -John ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; toy https server ;; (eval-when (load eval compile) (require 'cl+ssl) (require 'bordeaux-threads) (require 'usocket)) (defparameter *keyfile* "./server.key") (defparameter *certfile* "./server.crt") ;; the bad server associates an SSL with the client inside a ;; worker thread, not inside the main thread (defun test-https-server (&key (port 8080) (cert *certfile*) (key *keyfile*)) (format t "~&SSL server listening on port ~d~%" port) (let ((server (usocket:socket-listen "127.0.0.1" port :backlog 100))) (loop (let* ((socket (usocket:socket-accept server)) (stream (usocket:socket-stream socket))) ;; launch response in a separate thread (bordeaux-threads:make-thread (lambda () ;; MAKE SSL HERE (let ((client (cl+ssl:make-ssl-server-stream stream :external-format '(:iso-8859-1 :eol-style :lf) :certificate cert :key key))) (unwind-protect (progn (format client "HTTP/1.0 200 OK~%") (format client "Server: SSL-TEST/1.1~%") (format client "Content-Type: text/plain~%") (terpri client) (format client "HELLO WORLD") (finish-output client)) ;; maybe apachebench is surprised by this close? (close client))))))))) On Apr 3, 2011, at 3:26 AM, David Lichteblau wrote: > Hi, > > Quoting David Lichteblau (david(a)lichteblau.com) >> Quoting JTK (jetmonk(a)gmail.com) > [...] >>> I've appended a test case in which I think I may have found the source of >>> hunchentoot's problems with ssl and concurrency. >> >> Does the attached patch help? > > I have committed an updated version of the patch to CL+SSL CVS now. > > This version works now because it avoids use of buggy code in the > bordeaux-threads backend for SBCL. > > It passes ab testing on SBCL for me; I haven't rechecked other Lisps > yet. > >> It's done from Lisp, and should really be done directly in C, similar to >> the th-lock.c sample code from OpenSSL. But let's first check whether >> it addresses the issue at all. > > If anyone is interested in developing a C version, let's move to that > ultimately. But I've decided to commit the Lisp version for now, > because I think that correctness should come before performance in this > case. > > > d. > > _______________________________________________ > cl-plus-ssl-devel mailing list > cl-plus-ssl-devel(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/cl-plus-ssl-devel

1 0

[hunchentoot-devel] Support for PUT and DELETE HTTP Verbs?
by G Horiske 01 Apr '11

01 Apr '11

I'm considering using HT as a back-end RESTful web service for an Adobe Flex client. Does HT support the PUT and DELETE HTTP verbs? If so, how does one use them? I see from the docs that the "define-easy-handler has a "request-type" parameter that accepts "GET", "POST", or "BOTH". Is there any chance "PUT" and "DELETE" have been added recently? Thanks in advance.

3 2

[hunchentoot-devel] Fwd: (forw) Re: [cl-plus-ssl-devel] ssl crashing in hunchentoot. A solution?
by Hans Hübner 01 Apr '11

01 Apr '11

Here is another patch from David, can someone please test on SBCL multithreaded? Thanks! Hans ---------- Forwarded message ---------- From: David Lichteblau <david(a)lichteblau.com> Date: Fri, Apr 1, 2011 at 4:28 PM Subject: Re: (forw) Re: [cl-plus-ssl-devel] ssl crashing in hunchentoot. A solution? To: Hans H?bner <hans(a)huebner.org> Quoting Hans H?bner (hans(a)huebner.org) > Zugegeben, ich habe nur CCL getestet, für SBCL hilft es wohl nicht :/ Ein letzter blinder Versuch (alternativ muesste ich doch selbst testen): >From 207a12ea7053643ef53683907e6d894d8ecb5ecd Mon Sep 17 00:00:00 2001 From: David Lichteblau <david(a)lichteblau.com> Date: Fri, 1 Apr 2011 16:26:19 +0200 Subject: [PATCH 2/2] Fix thread id computation --- ffi.lisp | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/ffi.lisp b/ffi.lisp index d773bef..3049c7f 100644 --- a/ffi.lisp +++ b/ffi.lisp @@ -392,11 +392,17 @@ will use this value.") (bt:release-recursive-lock lock)))) (defvar *threads* (trivial-garbage:make-weak-hash-table :weakness :key)) +(defvar *thread-counter* 0) +(defparameter *initialization-lock* + (bordeaux-threads:make-recursive-lock "SSL initialization")) (cffi:defcallback threadid-callback :unsigned-long () (let ((self (bt:current-thread))) (or (gethash self *threads*) - (setf (gethash self *threads*) (hash-table-count *threads*))))) + (setf (gethash self *threads*) + (bordeaux-threads:with-recursive-lock-held + (*initialization-lock*) + (incf *thread-counter*)))))) (defun initialize (&key (method 'ssl-v23-method) rand-seed) (setf *locks* (loop @@ -415,9 +421,6 @@ will use this value.") (ssl-ctx-set-default-passwd-cb *ssl-global-context* (cffi:callback pem-password-callback))) -(defparameter *initialization-lock* - (bordeaux-threads:make-recursive-lock "SSL initialization")) - (defun ensure-initialized (&key (method 'ssl-v23-method) (rand-seed nil)) "In most cases you do *not* need to call this function, because it is called automatically by all other functions. The only reason to -- 1.7.0

3 3

[hunchentoot-devel] ssl crashing in hunchentoot. A solution?
by JTK 01 Apr '11

01 Apr '11

I hope it is OK to send this to both the hunchentoot and cl+ssl lists. I think it is relevant to both. I've appended a test case in which I think I may have found the source of hunchentoot's problems with ssl and concurrency. Code containing good and bad toy servers is attached at end of email. Apologies in advance if this is way out in left field. Here's my explanation - the comments in the code explain it in greater detail. It appears that if you run a main listener thread and turn a client connection into an SSL socket using cl+ssl:make-ssl-server-stream within the main listener, and pass this ssl-socket to a child thread, then things work. However, if you pass the raw socket to the child thread, and the child thread itself runs cl+ssl:make-ssl-server-stream, then SSL can fail and crash lisp if there are concurrent connections. It seems that hunchentoot passes the job of running cl+ssl:make-ssl-server-stream to child threads, as described in the comments to the code. I'd be interested if people agree with the diagnosis. I'm not really clear on what the broader repercussions of moving the initialization are, so I haven't patched hunchentoot. (plus I might not have more time for a while). John

3 2