November 2006 - tbnl-devel@common-lisp.net

[hunchentoot-devel] Re: Missing functions in Hunchentoot 0.4.10?
by Edi Weitz 21 Nov '06

21 Nov '06

Hi Dominic! Please use the mailing list. On Sun, 19 Nov 2006 22:40:43 +0000, "lisp" <lisp(a)spikeisland.com> wrote: > I've just downloaded the latest Hunchentoot 0.4.10 but noticed that > there seem to be a couple of functions missing: > > The following functions are undefined: HUNCHENTOOT::OCTETS-TO-STRING > which is referenced by HUNCHENTOOT:URL-DECODE > HUNCHENTOOT::STRING-TO-OCTETS which is referenced by > HUNCHENTOOT::COMPUTE-LENGTH and HUNCHENTOOT:URL-ENCODE > > As a result of this (or something else) the test samples aren't > working. Looks like you're using an old version of FLEXI-STREAMS. See the note about library versions here: http://weitz.de/hunchentoot/#install > As an aside, do you know if anyone is looking at whether UCW can be > run on top of Hunchentoot? Don't know. Have you asked on their mailing list? Cheers, Edi.

2 2

[hunchentoot-devel] serve-event support for SBCL and CLISP
by Luís Oliveira 20 Nov '06

20 Nov '06

Hello, I've added a new keyword argument :MODE to START-SERVER that can be one of :THREADED or :SERVE-EVENT and implemented serve-event support for SBCL and CLISP. This is a preliminary patch. No real-world testing or documentation yet. It might however already be useful for people doing development on, e.g., SBCL/OSX. In CLISP's case, you have to run (hunchentoot:clisp-serve-events) to for the server(s) to actually run and handle requests. By the way, should anyone deploy a server using serve-event, make sure to run it behind a reverse proxy such as Apache's mod_proxy or Pound otherwise a client with a slow connection will stall all the other clients. Also, regarding CLISP, it complains about WRITE-SEQUENCEing a sequence of (unsigned-byte 8) to a flexi-stream with a character element-type. See the START-OUTPUT function in headers.lisp towards the end. Not sure if that was the best fix or whether it should be conditionalized for CLISP. Comments welcome. -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/

2 4

Re: [hunchentoot-devel] *http-error-handler* behaviour
by Edi Weitz 19 Nov '06

19 Nov '06

On Fri, 17 Nov 2006 07:56:02 +0100, "Marijn Haverbeke" <marijnh(a)gmail.com> wrote: [Please use the mailing list.] > I hadn't seen aux-request-value yet -- that would definitely be > better than a dynamic variable. But I'm still not crazy about the > way the *http-error-handler* just kicks in when the headers are > being sent. All the other stuff in my app neatly fetches an output > stream, and then writes its body to it, but for non-200 pages the > library just refuses to let me do this. (Which means I can't use my > very nice function for filling and writing a template in this case.) > Seems a bit arbitrary. It was intended as a simple solution at a time when there was no customized error handling at all: http://common-lisp.net/pipermail/tbnl-devel/2005-March/000197.html I still think it is OK as far as it is in line with the way "normal" content is handled in Hunchentoot. The only real difference I can see is that you can't write directly to a stream. Why you can't use your nice function escapes me, though. You could capture its output in a string stream, couldn't you? If you come up with a solution that obviously improves the current situation, I'd be happy to include your patches. Bonus points if the new stuff is compatible with the old code. And note that a patch which changes user-visible behaviour should include documentation as well - I might otherwise reject it. Cheers, Edi.

2 4

[hunchentoot-devel] Socket timeout issues
by Toby 18 Nov '06

18 Nov '06

Hi all I'm having some trouble with connection timeouts. I'm writing an AJAX application where I need to push values from the server to the client at random times (it's a chat application.) To avoid polling the server every second, I wrote it like this, to exploit hanging connections: - the client opens a connection with XMLHttpRequest - the server doesn't respond (the handler waits on a condition variable), keeping the connection hanging - when the server has data to send (new lines of chat) it does so, and: - the client receives the data, closes the connection and starts over - if the connection timeouts (on either end), the client starts over This method worked beautifully on SBCL + TBNL + mod_lisp2 + Apache2, on any browser. It gave me the lowest possible lag (equal to the network latency) with very little network overhead. Now on SBCL 0.9.17 + Hunchentoot 0.4.9 I'm having trouble with the Opera browser (and possibly others: I've only tested it on Opera and Firefox.) By the way, Firefox doesn't seem to have any connection timeout: I've seen it receive a response after a (sleep 1800), without any packet sent by either side in half an hour! Anyway I've used a sniffer to understand what's going on. Usually, when the connection times out client-side, these packets are exchanged: C -> S ACK FIN ( C <- S ACK ) C <- S ACK FIN C -> S ACK This way both sides agree that the connection is over. But on SBCL + Hunchentoot vs. Opera, after 2 minutes the client times out and I see the following exchange: C -> S ACK FIN C <- S ACK I can't make sense of it: why does the server acknowledge the FIN packet but fail to send its own? As a result the client thinks it's almost over... enough not to accept any data from the same connection again, but not enough to stop spinning and fire the XMLHttpRequest callback (which would open a new connection.) Even worse, the server thinks it's still open, so it will send any new data on that wire. But the data won't be accepted by the client, which will send a RST as soon as it sees stuff coming from a 'dead' connection. Help! Toby PS: All these tests were done with :read-timeout nil Maybe I should set it to 119 and forget about it :-)

2 2

[hunchentoot-devel] *http-error-handler* behaviour
by Marijn Haverbeke 16 Nov '06

16 Nov '06

Hi, The way in which reply bodies are currently made when a non-200 response code is set seems a little inconvenient. When no *http-error-handler* is set, a standard body is given, and when a handler is set it is expected to generate a body. I was creating a system where bad requests can be signalled with conditions, which contain an error message that should be given to the user, and which I was going to render by first calling send-headers and then writing an html-templates template to the resulting stream... but my efforts were frustrated by the fact that send-headers calls start-output, which dumbly generates a body when an error code is set, and the *http-error-handler* is a rather clumsy way to hook into this -- it doesn't know anything about the context of the error, and communicating to it by using a dynamic variable is rather ugly. Maybe an extra argument to (setf return-code) which suppresses the generating of a standard body would be a nicer way to allow people to customize their error messages. (Or maybe having an accessors setf form take different arguments from its normal form is very bad style. -- Another way could be to leave response-code the way it is, default to not replacing the body in case of non-200 code, and add another function which sets the response-code and also enables the body-replacing functionality for that request.) Regards, Marijn Haverbeke

2 1

[hunchentoot-devel] header(s)-out
by Robert J. Macomber 12 Nov '06

12 Nov '06

[oops, originally sent this to tbnl@... hope that's not a different list or person. Hasn't bounced in any case.] There seems to be some small bugs involving header-out. The accessor is documented to take both keywords and strings, but neither works reliably because: The reader will use string-equal if given a string and eq if given a symbol, because of the assoc shadowing in the hunchentoot package. This will find what it's looking for if it's given a string or if it's given a keyword _and_ the header entry was originally created with a keyword. The writer will do the same, so it's possible to have two different entries for any given header in the headers-out list, one for a string entry and one for a symbol. (The code _looks_ like it does the right thing, but if "name" is a symbol, it calls the symbol-specialized version of assoc which ignores its test argument). Finally, if a header-out entry is created with a symbol, response generation fails because it uses write-string (or write-line with mod-lisp) to output the header name, and they don't take string designators. Here's a patch which normalizes things so the keys of this alist are always strings and the lookup is always done with strings. It uses string-capitalize to convert keywords when storing in order to make them have conventional HTTP header capitalization. --- hunchentoot-0.4.8.orig/reply.lisp 2006-11-05 15:55:06.000000000 -0700 +++ hunchentoot-0.4.8/reply.lisp 2006-11-08 15:40:15.000000000 -0700 @@ -115,7 +115,7 @@ (defun header-out (name &optional (reply *reply*)) "Returns the current value of the outgoing http header named NAME. NAME should be a keyword or a string." - (cdr (assoc name (headers-out reply)))) + (cdr (assoc (string name) (headers-out reply)))) (defun cookie-out (name &optional (reply *reply*)) "Returns the current value of the outgoing cookie named @@ -129,7 +129,8 @@ created." (with-rebinding (name reply) (with-unique-names (place) - `(let ((,place (assoc ,name (headers-out ,reply) :test #'string-equal))) + `(let* ((,name (if (stringp ,name) ,name (string-capitalize ,name))) + (,place (assoc ,name (headers-out ,reply)))) (cond (,place (setf (cdr ,place) ,new-value)) -- Robert Macomber tbnl(a)rojoma.com

2 1

[hunchentoot-devel] real-remote-addr and proxy chains
by Robert J. Macomber 07 Nov '06

07 Nov '06

The real-remote-addr function currently returns the value of the x-forwarded-for header if it's set, or remote-addr if it's not. In the case of chains of proxies, this gives unexpected results as each proxy appends the address it's proxying for onto the end of the list. Since I imagine this function is intended to be used in situations where Hunchentoot is sitting behind proxies of its own, I've written a function to split things up to give a particular entry in this chain. Most of the time, I imagine you'd just want the address added by the closest proxy but if (for example) you're behind mod_proxy behind squid, this function can tell you the address of the agent that hit the squid server. I'm not actually quite sure what to call this function. Originally at my local site I'd called it "real-remote-addr", replacing Hunchentoot's own function, but the name's not sitting easily with me. For one thing, it's got a different API since it takes the number of hops along the proxy chain to look, and for another it returns NIL if you fall off the end of the chain (I'm not quite sure that's correct, but it seems to me to fail better than DWIMly returning the last available address, since values supplied by the user's original request are useless anyway). Anyway, here it is, still with the name real-remote-addr. (defvar *proxy-count* 0 "The length of the chain of server-side proxies in front of Hunchentoot.") (defun real-remote-addr (&optional (nth *proxy-count*) (request *request*)) "Returns the address of the NTH host in the chain of proxies that set the X-Forwarded-For header. 0 is the address of the last proxy (or the client, if there are no proxies), 1 the address of the second proxy (or client if there is only one), and so forth." (if (zerop nth) (remote-addr request) (let* ((proxies (loop with xff = (header-in :x-forwarded-for request) for pos = 0 then (1+ comma) for comma = (position #\, xff :start pos) collect (string-trim " " (subseq xff pos comma)) while comma)) (position (- (length proxies) nth))) (if (minusp position) nil (nth position proxies))))) -- Robert Macomber tbnl(a)rojoma.com

3 7

[hunchentoot-devel] Problems with file uploads and mod_lisp
by Edi Weitz 07 Nov '06

07 Nov '06

Hi! I just realized that Hunchentoot in its current form has problems with file uploads if it's used behind mod_lisp. I could reproduce this with Apache 2 as well as Apache 1 and with LispWorks (but only on Linux, not on Windows) as well as with SBCL. To try this yourself, start the demo that comes with Hunchentoot and upload a couple of pictures (I used JPEGs) in the file upload section. Click on the uploaded pictures and view them. Symptoms that I see: - files are sometimes uploaded only partly - you see an Apache error message after an upload is finished (which usually means that mod_lisp can't talk to Lisp) - server gets completely unresponsive after some time It seems this is not deterministic, though. I've had situations where I uploaded the same picture a couple of times, and sometimes the whole file reached the server, sometimes only a part of it. One of the reasons for this that I could observe is that when rfc2388.lisp reads characters from the content stream, READ-CHAR sometimes returns NIL although the whole file hasn't been transferred yet (i.e. there should be more input available). I'm writing this because I haven't found the reason for this behaviour yet and I'll probably be too busy in the next days to do something about it. Obviously, I'd be happy if someone else would debug this and find a solution. Meanwhile, maybe you shouldn't use Hunchentoot with mod_lisp for production servers. The stand-alone version doesn't seem to be affected, though. Cheers, Edi.

2 4

[hunchentoot-devel] Confusing Hunchentoot behaviour
by Marijn Haverbeke 06 Nov '06

06 Nov '06

Hello, I am going to try to report something that looks like a bug, or maybe multiple bugs, but right now I am so bewildered that I'm not sure it's not something stupid I am doing... It all started when I was playing around with a lisp image and a Hunchentoot server, trying to get the hang of the library. When I added some things to *dispatch-table*, the server seemed to completely ignore me. (By the way, this is SBCL 0.9.18 on Linux behind mod_lisp). This was a bit frustrating. After poking the source for a bit, and making dispatch-request log the table it was getting, it turns out the server was using a value different from the current top-level *dispatch-table*. I assumed this was because it bound the result of calling *meta-dispatcher* to its own *dispatch-table* binding and apparently never left process-connections. I was about to write an email to this list about how this might not be the most desirable behaviour, but decided to test it one more time to make sure I wouldn't make a fool out of myself. And what do you know, after restarting my lisp process, starting a new server, and then messing with the *dispatch-table* it suddenly DID take my changes into account. 'What the hell', was what I thought. But it got even better. Refreshing the test page a few times showed that sometimes the changed *request-table* was used, but other times it would use the old one! I was about to dismiss Hunchentoot as a bug-ridden pile of weidness at this point. But while I was writing this incoherent complaint I finally got it -- obviously process-request is called once for every THREAD, and since the server will keep a bunch of (4 on my setup) long-lived listening threads when communicating with mod_lisp, strange results will occur when each of these has a different dispatch-table -- the same client will see different behaviour from the page. I'd say this is not a desirable feature. A good fix would be to just call *meta-dispatcher* for every call to dispatch-request, so that you've always got the most recent dispatch-table and the programmer can fiddle with that variable from the REPL and immediately see the result. Unless someone goes and writes a really complicated and expensive meta-dispatcher this shouldn't be a problem. -- As far as I can see no other parts of the code refer to the dynamic *dispatch-table* var, so losing that binding shouldn't be a problem. Let me know what you think. Marijn Haverbeke

2 4

[hunchentoot-devel] Re: Huchentoot
by Edi Weitz 02 Nov '06

02 Nov '06

Hi Bill! Thanks. I have no plans to add that to the code base, but I'm forwarding your email to the mailing list, so people can find it and try it out if they want. Cheers, Edi. On Wed, 1 Nov 2006 09:17:18 -0800, "William Bland" <doctorbill(a)gmail.com> wrote: > On 10/31/06, William Bland <doctorbill(a)gmail.com> wrote: >> >> Doh! Thanks Edi, of course that was it - sorry to have bothered you. >> >> I still can't get it to compile in my environment (SBCL on a MacBook) >> because it complains about the lack of threads, but I'm glad to at >> least know what I was doing wrong there! >> > > Hi again, > > I have Hunchentoot working now, in SBCL, on my MacBook. The changes I > had to make to port-sbcl.lisp were: > > 32c32 > < #-(and :sb-thread :sb-unicode) > --- >> #-sb-unicode > 34c34,38 > < (error "This library needs a version of SBCL with Unicode and > thread support.")) > --- >> (error "This library needs a version of SBCL with Unicode support.")) >> >> #-sb-thread >> (eval-when (:compile-toplevel :load-toplevel :execute) >> (warn "Without thread support, this library is only useful for development.")) > 67,69c71,74 > < (sb-thread:make-thread (lambda () > < (apply function args)) > < :name name)) > --- >> #+sb-thread(sb-thread:make-thread (lambda () >> (apply function args)) >> :name name) >> #-sb-thread(apply function args)) > > Of course it's not really useful for a production web server, but it's > great to be able to run the thing on my laptop for development before > I deploy to my server. > > I've tested all of the demos in hunchentoot-test. They appear to work > perfectly in my non-threaded environment. > > Cheers, > Bill. > -- > William Bland: http://www.abstractnonsense.org/ > Lisp documentation: http://www.lispdoc.com/

1 0