tbnl-devel July 2008

tbnl-devel@common-lisp.net

16 participants
14 discussions

[hunchentoot-devel] with-recursive-lock-held
by Nick Allen 31 Jul '08

31 Jul '08

Hello! Is the "ediware" location of the hunchentoot source the recommended version for deployment? or should we be using the tarball linked off the documentation or the "hans" version? I'm wondering because WITH-RECURSIVE-LOCK-HELD seems to have disappeared out of "lispworks.lisp", which seems to have introducing some bugs, which seems to me to suggests to me that the "ediware" repo is either out of date or too bleeding edge... take care Nick

2 4

[hunchentoot-devel] Hunchentoot behind apache http proxy (performance)
by Andrei Stebakov 25 Jul '08

25 Jul '08

The moment came when I had to move all of my site from being partially static http web pages to pages generated (or served from cached files) by Hunchentoot. The overall performance noticeably degraded in terms of response time. Even in cases when HT does nothing but serves a static file from its cache the response time increased from 0.3 sec to 1.5 sec (tested with httperf). I am running Apache 2 on Ubuntu (everything latest) with sbcl 1.0.18. I understand that when ht running behind the proxy the response time should increase. I wonder what can I do to optimize. Is it time to run HT as a standalone server? I still have a bunch of pages that I need to serve statically and lately my sbcl had problems with GC so it crashes time after time from heap exhaustion, so I am a bit uncomfortable to switch completely to Hunchentoot. I wonder how do you guys solve those problems? Thank you, Andrew

2 3

[hunchentoot-devel] Redirecting preserving the session?
by Andrei Stebakov 24 Jul '08

24 Jul '08

When I use redirect method it redirects to a new url and in the handler for the url I have no way of knowing the session of the url from where it was redirected. How can I send cookies along with redirecting to a new url?

2 3

[hunchentoot-devel] hunchentoot-cgi and the new hunchentoot regime
by Cyrus Harmon 22 Jul '08

22 Jul '08

So... I'm trying to convert my hunchentoot-cgi over to the new hunchentoot and running into some problems. My old approach was to do: (progn (let* ((return-code (tbnl::return-code)) (reason-phrase (reason-phrase return-code)) (first-line (format nil "HTTP/1.1 ~D ~A" return-code reason-phrase))) (write-sequence (map 'list #'char-code first-line) out) (write-sequence tbnl::+crlf+ out) (tbnl::maybe-write-to-header-stream first-line)) (setf tbnl::*headers-sent* t) (setf (tbnl::content-type) nil) (sb-ext::run-program path nil :output out :environment env) nil) but this doesn't seem to work anymore. My new approach is to try to use send-headers, but with some hackery to make it optionally more cgi-friendly: Index: headers.lisp =================================================================== --- headers.lisp (revision 3423) +++ headers.lisp (working copy) @@ -70,6 +70,8 @@ (:method (key value) (write-header-line key (princ-to-string value)))) +(defparameter *cgi-hack* nil) + (defun start-output (&key (content nil content-provided-p) (request *request*)) "Sends all headers and maybe the content body to @@ -84,7 +86,8 @@ ;; Read post data to clear stream - Force binary mode to avoid OCTETS-TO-STRING overhead. (raw-post-data :force-binary t) (let* ((return-code (return-code)) - (chunkedp (and (server-output-chunking-p *server*) + (chunkedp (and (not *cgi-hack*) + (server-output-chunking-p *server*) (eq (server-protocol request) :http/1.1) ;; only turn chunking on if the content ;; length is unknown at this point... @@ -196,12 +199,14 @@ ;; write all headers from the REPLY object (loop for (key . value) in (headers-out) when value - do (write-header-line (as-capitalized-string key) value)) + do (unless (and *cgi-hack* (equal key :content-type)) + (write-header-line (as-capitalized-string key) value))) ;; now the cookies (loop for (nil . cookie) in (cookies-out) do (write-header-line "Set-Cookie" (stringify-cookie cookie))) ;; all headers sent - (write-sequence +crlf+ *hunchentoot-stream*) + (unless *cgi-hack* + (write-sequence +crlf+ *hunchentoot-stream*)) (maybe-write-to-header-stream "") ;; access log message (when-let (access-logger (server-access-logger *server*)) The motivation for this hack being that the CGI script wants to send the content-type header, so we can't finish the headers here when running a CGI script. Then I can do: (let* ((tbnl::*cgi-hack* t) (stream (flexi-streams:make-flexi-stream (tbnl:send-headers) :external-format tbnl::+latin-1+))) (sb-ext::run-program path nil :output stream :environment env)) and this sort of works, but, 1) it's kind of a hack and 2) it either exacerbates the timeout situation or there's some other problem where the first request is handled promptly, but subsequent requests to 10 seconds or so while something times out. Not sure why this would be. any thoughts on a good way to handle this? thanks, Cyrus

3 8

[hunchentoot-devel] redirect method works in IE but not in Firefox
by Andrei Stebakov 22 Jul '08

22 Jul '08

When I use redirect method at the end of ht handler (which I used for a long time before) suddenly it stopped working in Firefox (though it works in IE). Firefox doesn't receive the http response with code 302. It started to happen after I added quite a few more drakma's http-requests with cookies in the handler. Any idea why it might be happening? Thank you, Andrew

3 3

[hunchentoot-devel] multiple POST/GET values
by Jaap de Heer 21 Jul '08

21 Jul '08

Howdy, I ran into trouble with Hunchentoot trying to process the result of a form containing a select multiple: <select multiple name="foo-or-bar"> <option value="foo">Foo <option value="bar">Bar </select> The user can select both foo and bar, which results in both being submitted. In the POST request, this looks something like: "other-field-1=x&foo-or-bar=foo&foo-or-bar=bar&other-field-2=x" Hunchentoot will pick up on only one of them, i.e. (get-post-parameter "foo-or-bar") will be either "foo" or "bar" but not both; it should be both. I've baked a patch for this that will make (get-post-parameter) et al return a list containing all submitted values for this field, if there is more than 1 value; otherwise, it just returns the string as usual. My Lisp fu is limited, and this is only one of a few possible solutions, so feel free to write a better one ;-) BTW. I suppose the same problem and fix applies for multiple checkboxes with the same names, and (though this may be invalid) even multiple <input> fields with the same name. Cheers, Jaap --- util.lisp.org 2008-07-21 20:29:31.000000000 +0200 +++ util.lisp 2008-07-21 20:29:57.000000000 +0200 @@ -168,6 +168,25 @@ (loop for code across (md5:md5sum-sequence string) do (format s "~2,'0x" code)))) +(defun group-by (lst &key (key #'identity) (test #'eq)) + "Transform a sorted list into a list of lists, grouping together elements with equal key." + (labels ((lp (lst result current-group) + (cond ((null lst) (nreverse (if (null current-group) + result + (cons (nreverse current-group) result)))) + ((or (null current-group) + (funcall test + (funcall key (first lst)) + (funcall key (first current-group)))) + (lp (rest lst) + result + (cons (first lst) current-group))) + (t + (lp (rest lst) + (cons (nreverse current-group) result) + (list (first lst))))))) + (lp lst nil nil))) + (defun escape-for-html (string) "Escapes the characters #\\<, #\\>, #\\', #\\\", and #\\& for HTML output." (with-output-to-string (out) @@ -281,12 +300,18 @@ &optional (external-format *hunchentoot-default-external-format*)) "Converts a list FORM-URL-ENCODED-LIST of name/value pairs into an alist. Both names and values are url-decoded while doing this." - (mapcar #'(lambda (entry) - (destructuring-bind (name &optional value) - (split "=" entry :limit 2) - (cons (string-trim " " (url-decode name external-format)) - (url-decode (or value "") external-format)))) - form-url-encoded-list)) + (mapcar (lambda (x) (cons (caar x) + (if (> (length x) 1) + (mapcar #'cdr x) + (cdar x)))) + (group-by (sort (mapcar #'(lambda (entry) + (destructuring-bind (name &optional value) + (split "=" entry :limit 2) + (cons (string-trim " " (url-decode name external-format)) + (url-decode (or value "") external-format)))) + form-url-encoded-list) + #'string< :key #'car) + :key #'car :test #'equal))) (defun url-encode (string &optional (external-format *hunchentoot-default-external-format*)) "URL-encodes a string using the external format EXTERNAL-FORMAT."

3 4

[hunchentoot-devel] thread pool benchmark results
by Anton Vodonosov 18 Jul '08

18 Jul '08

Hello. I have set up a little benchmark to test ithow thread pool may improve the hunchentoot performance. In two words it looks useful. Details are below. I'm running hunchentoot on a linux VPS with 360 MB of memory, SBCL 1.0.11. It serves little html-template page, the template is filled from an in-memory object. Testing is performed by the AbacheBench tool: ab -c 10 -n 10000 http://localhost:4242/view-order?order-id=1 -n 10000 means total number of requests is 10000 -c 10 means that the requests are issued by 10 concurrent threads Result for the unmodified hunchentoot 0.15.7 was 163 requests per second. For slightly modified version with thread pool it was 1295 request per second. Full AbacheBench reports for the both versions are attached. Patch for port-sbcl.lisp with the simple thread pool implementation is also attached. It is against 0.15.7, sorry it's not for the current development version of hunchentoot, but I created this thead pool code about half a year ago. Best regards, -Anton

2 2

[hunchentoot-devel] form array
by Jens Teich 16 Jul '08

16 Jul '08

Maybe this is only a problem of my wrong handling of html forms. I try to set up a group of checkboxes like this: <form action='test.html'> <input type='checkbox' name='array[]'> <input type='checkbox' name='array[]'> <input type='checkbox' name='array[]'> ... I try to get the user input into a Lisp array with an easy-handler but get only an empty array. When clicking the submit button I see an URL like test.html?array%5B%5D=on... where I expect test.html?array[0]=on... If this is already my problem it has nothing to do with huchentoot, but I would appreciate any kind of help ... Thanks Jens

2 2

[hunchentoot-devel] SLIME + SBCL + Hunchentoot on Windows XP - how do I stop the server
by Mathias Dahl 14 Jul '08

14 Jul '08

Hi! I have got SLIME + SBCL + Hunchentoot working on Windows XP, serving a small web app. I also access a small SQLITE database via CLSQL, which works too. All is well but the fact that once I started Hunchentoot from within SLIME, it locks up SLIME. I assume this is because there is no thread support in SBCL under Windows. However, I can live with this if only I knew a way to stop Hunchentoot. Then I could evaluate whatever new or changed stuff on the REPL and then start the server again. I tried C-c C-c (Interrupt Command) but that killed SLIME and SBCL). Any hints? /Mathias

2 6

[hunchentoot-devel] feature request
by Cyrus Harmon 09 Jul '08

09 Jul '08

Currently, if a file is setup for logging and some other program modifies the file, subsequent log messages fail with a message like this: [2008-07-08 16:56:00 [ERROR]] Error while processing connection: I/O timeout reading #<SB-SYS:FD-STREAM for "a socket" {136DDCD1}>. It would be nice if we trapped this error and reopened the file for appending at this point instead of just timing out. Not sure how involved this is though. Figured I'd record the intention here for posterity's sake in case I never get around to trying to do this. thanks, Cyrus

2 2

[hunchentoot-devel] Problem with SBCL 1.0.18
by Andrew Peterson 08 Jul '08

08 Jul '08

Today, I upgraded to SBCL 1.0.18 and Hunchentoot 0.15.7 from SBCL 1.0.12 and Hunchentoot 0.15.2. Unfortunately, when SBCL compiles I got this error: erred while invoking #<COMPILE-OP NIL {1002F3BE11}> on #<CL-SOURCE-FILE "port-sbcl" {10031BF5B1}> [Condition of type ASDF:COMPILE-FAILED] I also got this error when I returned to Hunchentoot 0.15.2. I returned to SBCL 1.0.12 and Hunchentoot works in both versions (0.15.2 and 0.15.7) andy peterson.

3 6

[hunchentoot-devel] Test for running?
by Robin Lee Powell 07 Jul '08

07 Jul '08

So I'm writing code that'll be largely "setup dispatch table, run hunchentoot". I don't really want to do anything after hunchentoot is launched other than, well, hunchentoot, but it seems to thread itself off (fine) and then the REPL terminates (not so fine). I'd like to loop around "Is Hunchentoot still alive?", but I don't know how to test for whether that thread is still running, or if that's even the right kind of test. This is SBCL on Linux, in case it matters. Thanks. -Robin -- Lojban Reason #17: http://en.wikipedia.org/wiki/Buffalo_buffalo Proud Supporter of the Singularity Institute - http://singinst.org/ http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/

3 2

[hunchentoot-devel] socket timeout errors
by Jeff Cunningham 06 Jul '08

06 Jul '08

Now that I've cleaned the Google forest out of my logs I can see something else that must be wrong. 20 seconds after *every* new set of page GETs in the access_log there is a socket timeout in the error_log: 127.0.0.1 (76.64.38.115) - [2008-07-05 05:54:09] "GET /ts7200.html HTTP/1.1" 200 20220 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0" 127.0.0.1 (76.64.38.115) - [2008-07-05 05:54:09] "GET /images/jandm.ico HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0" [2008-07-05 05:54:29 [ERROR]] Error while processing connection: I/O timeout reading #<SB-SYS:FD-STREAM for "a socket" {14564111}>. I believe this is new behavior since I upgraded hunchentoot and switched from mod_lisp to mod_proxy (with which I am unfamiliar), in case that might have anything to do with it. I don't have any idea what might be going on here. The pages are complete and don't appear to be timing out. Thanks. --Jeff

2 2

[hunchentoot-devel] googlebot revisitation rate excessive?
by Jeff Cunningham 05 Jul '08

05 Jul '08

Has anyone else noticed excessive visitation of pages served dynamically by Hunchentoot? On a couple of small sites virtually every page gets hit every five minutes by http://www.google.com/bot.html, regular as clockwork. Normally, I setup robots.txt to allow google to walk these sites but unless I can think of a way to tell them to back off to once a week or something I'm about to Disallow them. They're just little private sites and Google ends up being 99% of the traffic. It's silly. --Jeff

6 14

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

tbnl-devel July 2008