- tbnl-devel@common-lisp.net - mailman3.common-lisp.net

Re: [hunchentoot-devel] Trouble with finish-output in process-connection
by Hans Hübner 04 Apr '11

04 Apr '11

Hi Ilya, please Cc the list in all emails concerning Hunchentoot. Reply below. On Mon, Apr 4, 2011 at 10:14 AM, Ilya Perminov <iperminov(a)dwavesys.com> wrote: > Hi Hans, > > Why is CLOSE always called with :ABORT T? It seems to me it is only > necessary when an IO error occurs and the connection have to be dropped. > If it is possible to change the code to use CLOSE without :ABORT T in > normal cases, it will be completely safe to use FORCE-OUTPUT inside of > the loop. I am not sure the change will resolve the "hanging in > FINISH-OUTPUT" problem, but it will make debugging easier. Ah, now I see what you mean. I will look at the code to find out why :ABORT is used later, but I suspect that there is no good reason to do that in non-error cases. > It would be really useful to find a simple way to reproduce the "hanging > in FINISH-OUTPUT" problem. If it is a SBCL bug, getting it fixed in SBCL > will be better than implementing a workaround. I agree with both points made. If there is a suitable workaround for older SBCL versions, it'd be even better. -Hans

1 0

[hunchentoot-devel] Trouble with finish-output in process-connection
by Andrey Moskvitin 04 Apr '11

04 Apr '11

Hi, A few days ago on the site lisper.ru I started using Hunchentoot from trunk. And found that the system stopped working in a few hours. I watched the behavior of the system during the day and found that some threads do not complete their work and hangs on "finish-ouptut" (in "process-connection"). When they accumulate enough, the work stops. I have this problem on Debian 64-bit with SBCL-1.0.47 and SBCL-1.0.37. I have not had this trouble when used Hunchentoot-1.1.1 After replacing the "finish-output" on "force-output" in the "process-connection" the problem was fixed for me. Andrey

3 7

[hunchentoot-devel] ssl crashing in hunchentoot. ssl patch works in CCL, but not SBCL on mac
by JTK 04 Apr '11

04 Apr '11

[cc'ed to hunchentoot list] I can confirm that ccl+hunchentoot on OS X 10.6 does not crash in ssl at a concurrency of 20 after 100,000 accesses (100% success in apache bench), whereas before it would crash after a few hundred. However, SBCL 1.0.47 is still freezing, but not coredumping. It just times out on the apachebench side, and in slime both the *inferior-lisp* and *slime-repl* stop responding, and the CPU is pegged at 100%. Using the toy server below, ab also causes an eventual freeze in sbcl but with CCL it mostly works, sometimes giving "SSL read failed - closing connection," which might be the fault of a premature close in the toy server (but I don't see how to do it otherwise, besides waiting until the client side closes). So this isn't a hunchentoot problem, most likely. Given that CCL+hunchentoot works perfectly maybe the problem is inside SBCL, or maybe just the Mac version of SBCL if other platforms now work. -John ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; toy https server ;; (eval-when (load eval compile) (require 'cl+ssl) (require 'bordeaux-threads) (require 'usocket)) (defparameter *keyfile* "./server.key") (defparameter *certfile* "./server.crt") ;; the bad server associates an SSL with the client inside a ;; worker thread, not inside the main thread (defun test-https-server (&key (port 8080) (cert *certfile*) (key *keyfile*)) (format t "~&SSL server listening on port ~d~%" port) (let ((server (usocket:socket-listen "127.0.0.1" port :backlog 100))) (loop (let* ((socket (usocket:socket-accept server)) (stream (usocket:socket-stream socket))) ;; launch response in a separate thread (bordeaux-threads:make-thread (lambda () ;; MAKE SSL HERE (let ((client (cl+ssl:make-ssl-server-stream stream :external-format '(:iso-8859-1 :eol-style :lf) :certificate cert :key key))) (unwind-protect (progn (format client "HTTP/1.0 200 OK~%") (format client "Server: SSL-TEST/1.1~%") (format client "Content-Type: text/plain~%") (terpri client) (format client "HELLO WORLD") (finish-output client)) ;; maybe apachebench is surprised by this close? (close client))))))))) On Apr 3, 2011, at 3:26 AM, David Lichteblau wrote: > Hi, > > Quoting David Lichteblau (david(a)lichteblau.com) >> Quoting JTK (jetmonk(a)gmail.com) > [...] >>> I've appended a test case in which I think I may have found the source of >>> hunchentoot's problems with ssl and concurrency. >> >> Does the attached patch help? > > I have committed an updated version of the patch to CL+SSL CVS now. > > This version works now because it avoids use of buggy code in the > bordeaux-threads backend for SBCL. > > It passes ab testing on SBCL for me; I haven't rechecked other Lisps > yet. > >> It's done from Lisp, and should really be done directly in C, similar to >> the th-lock.c sample code from OpenSSL. But let's first check whether >> it addresses the issue at all. > > If anyone is interested in developing a C version, let's move to that > ultimately. But I've decided to commit the Lisp version for now, > because I think that correctness should come before performance in this > case. > > > d. > > _______________________________________________ > cl-plus-ssl-devel mailing list > cl-plus-ssl-devel(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/cl-plus-ssl-devel

1 0

[hunchentoot-devel] Support for PUT and DELETE HTTP Verbs?
by G Horiske 01 Apr '11

01 Apr '11

I'm considering using HT as a back-end RESTful web service for an Adobe Flex client. Does HT support the PUT and DELETE HTTP verbs? If so, how does one use them? I see from the docs that the "define-easy-handler has a "request-type" parameter that accepts "GET", "POST", or "BOTH". Is there any chance "PUT" and "DELETE" have been added recently? Thanks in advance.

3 2

[hunchentoot-devel] Fwd: (forw) Re: [cl-plus-ssl-devel] ssl crashing in hunchentoot. A solution?
by Hans Hübner 01 Apr '11

01 Apr '11

Here is another patch from David, can someone please test on SBCL multithreaded? Thanks! Hans ---------- Forwarded message ---------- From: David Lichteblau <david(a)lichteblau.com> Date: Fri, Apr 1, 2011 at 4:28 PM Subject: Re: (forw) Re: [cl-plus-ssl-devel] ssl crashing in hunchentoot. A solution? To: Hans H?bner <hans(a)huebner.org> Quoting Hans H?bner (hans(a)huebner.org) > Zugegeben, ich habe nur CCL getestet, für SBCL hilft es wohl nicht :/ Ein letzter blinder Versuch (alternativ muesste ich doch selbst testen): >From 207a12ea7053643ef53683907e6d894d8ecb5ecd Mon Sep 17 00:00:00 2001 From: David Lichteblau <david(a)lichteblau.com> Date: Fri, 1 Apr 2011 16:26:19 +0200 Subject: [PATCH 2/2] Fix thread id computation --- ffi.lisp | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/ffi.lisp b/ffi.lisp index d773bef..3049c7f 100644 --- a/ffi.lisp +++ b/ffi.lisp @@ -392,11 +392,17 @@ will use this value.") (bt:release-recursive-lock lock)))) (defvar *threads* (trivial-garbage:make-weak-hash-table :weakness :key)) +(defvar *thread-counter* 0) +(defparameter *initialization-lock* + (bordeaux-threads:make-recursive-lock "SSL initialization")) (cffi:defcallback threadid-callback :unsigned-long () (let ((self (bt:current-thread))) (or (gethash self *threads*) - (setf (gethash self *threads*) (hash-table-count *threads*))))) + (setf (gethash self *threads*) + (bordeaux-threads:with-recursive-lock-held + (*initialization-lock*) + (incf *thread-counter*)))))) (defun initialize (&key (method 'ssl-v23-method) rand-seed) (setf *locks* (loop @@ -415,9 +421,6 @@ will use this value.") (ssl-ctx-set-default-passwd-cb *ssl-global-context* (cffi:callback pem-password-callback))) -(defparameter *initialization-lock* - (bordeaux-threads:make-recursive-lock "SSL initialization")) - (defun ensure-initialized (&key (method 'ssl-v23-method) (rand-seed nil)) "In most cases you do *not* need to call this function, because it is called automatically by all other functions. The only reason to -- 1.7.0

3 3

[hunchentoot-devel] ssl crashing in hunchentoot. A solution?
by JTK 01 Apr '11

01 Apr '11

I hope it is OK to send this to both the hunchentoot and cl+ssl lists. I think it is relevant to both. I've appended a test case in which I think I may have found the source of hunchentoot's problems with ssl and concurrency. Code containing good and bad toy servers is attached at end of email. Apologies in advance if this is way out in left field. Here's my explanation - the comments in the code explain it in greater detail. It appears that if you run a main listener thread and turn a client connection into an SSL socket using cl+ssl:make-ssl-server-stream within the main listener, and pass this ssl-socket to a child thread, then things work. However, if you pass the raw socket to the child thread, and the child thread itself runs cl+ssl:make-ssl-server-stream, then SSL can fail and crash lisp if there are concurrent connections. It seems that hunchentoot passes the job of running cl+ssl:make-ssl-server-stream to child threads, as described in the comments to the code. I'd be interested if people agree with the diagnosis. I'm not really clear on what the broader repercussions of moving the initialization are, so I haven't patched hunchentoot. (plus I might not have more time for a while). John

3 2

[hunchentoot-devel] SSL and concurrency crashes Lisp
by Daniel Brunner 31 Mar '11

31 Mar '11

Hi to everybody, I ran into the following problem with the SSL layer: I start my Lisp (CCL or SBCL, doesn't matter) and feed it with these lines: --8<-- (ql:quickload "hunchentoot") (defparameter *server* (hunchentoot:start (make-instance 'hunchentoot:ssl-acceptor :port 4711 :ssl-certificate-file #P"/home/dbr/local-cert.pem" :ssl-privatekey-file #P"/home/dbr/local-key.pem"))) --8<-- Then a SSL-ACCEPTOR is running and serving as wished. But when I use Apache Bench with some concurreny: > ab -f SSL2 -n 1000 -c 4 https://localhost:4711/ then the Lisp crashes and runs into the debugger. E.g. on CCL/Linux: --8<-- ? Unhandled exception 11 at 0x7f1843f1ebe9, context->regs at #x7f184390f428 Exception occurred while executing foreign code at lh_insert + 137 received signal 11; faulting address: 0x7f1843f8fa08 invalid permissions for mapped object --8<-- Or on SBCL/Linux: --8<-- CORRUPTION WARNING in SBCL pid 3549(tid 140737314223872): Memory fault at 0 (pc=0x7ffff4c6f940, sp=0x7ffff59ee280) The integrity of this image is possibly compromised. Continuing with fingers crossed. --8<-- I tested this scenario on different platforms: Clozure CL (1.7-dev-r14406M-trunk but it crashes on older versions as well) - 32 Bit on Windows Server 2003 - 64 Bit on Windows Server 2008 - 32 Bit on Ubuntu 10.04 - 64 Bit on Ubuntu 10.04 (libssl.0.9.8) SBCL 1.0.46 64 Bit on Ubuntu 10.04 (libssl.0.9.8) Used libraries/hunchentoot: - dists/quicklisp/software/cl+ssl-20101006-cvs/ - dists/quicklisp/software/hunchentoot-1.1.1/ Is this a known problem? Does anybody has an idea how to solve it? Kind regards, Daniel.

3 2

[hunchentoot-devel] session-value
by Haris 30 Mar '11

30 Mar '11

Hi. Can I increase session value with javascript/parenscript ? For example, I start the session, set a session value 'sum to 0 and then I want to increase its value on :onclick event ? Thanks

2 1

[hunchentoot-devel] Questions about using the same uri, but a different function for post and get methods.
by Jim Barrows 30 Mar '11

30 Mar '11

I have a uri "/my-uri", when a get request is made I want to return whatever information I have, and when a post request made, I want to update that information. I had thought that doing this: (define-easy-handler(get-business :uri "/business" :default-request-type :get)() (with-html-output-to-string (*standard-output* nil :prologue nil :indent t) (format t "{\"success\":true, \"data\": ~a}" (encode-json-plist-to-string (business::find-business))))) (define-easy-handler(post-business :uri "/business" :default-request-type :post) (business-name) (with-html-output-to-string( *standard-output* nil :prologue nil :indent t) (format t "post-business - ~a" business-name))) would accomplish that. However, the second handler is always called, for the get and the post. I'm new to Lisp, and was wondering what I'm missing. Any help would be greatly appreciated. -- James A Barrows

4 4

[hunchentoot-devel] any way to make upload more efficient?
by valgusk lalala 25 Mar '11

25 Mar '11

Hi there. I have just started with Hunchentoot. Being also a lisp newcomer i started out with reading tutorials and docs and this one mailing list, where I found some upload problems mentioned and getting interested in testing them out. After installation of hunchentoot (as a standalone server) on sbcl and trying to upload some files I got almost 100% cpu load on the host machine with just one file being uploaded. Then I tested it, hacking the rfc2388 module to only read from stream without processing or writing anything. I got a 50%+ cpu cload. The upload on apache/php is getting 5%. I saw that the lisp is equally heating the cpu just with stream reading-writing by making a separate file read-write test. It's an sbcl stream problem as i see it, nothing can be done here, I guess. It's acceptable for apache being held on this system, but i don't really want to run php alongside with lisp just for file uploads. So, do you think it would be possible to write some foreign module for just the multipart form-data processing and use it with Hunchentoot? Any way to make upload performance reasonable?

2 1