July 2004 - tbnl-devel@common-lisp.net - mailman3.common-lisp.net

[tbnl-devel] Handling HTTP errors
by Stefan Scholl 01 Mar '05

01 Mar '05

Hi! Function send-output in modlisp.lisp handles the response for different return codes. E. g. there's a "The requested URL ~A was not found on this server." for the 404 (+http-not-found+). How about letting the user change this behavior? One could explain the error to the user. Maybe in another language. Or think of all the funny 404 games. Remember the SGI babies? There could be different approaches: * Simple hash A hash with error code as key and handler as value. + Easy to implement + Can handle all errors (thrown by handlers _and_ when no dispatcher is found) - Not very flexible. Just checks for the code and not for any request data. * Error dispatcher Like the normal dispatcher, but checking for the right error output handler. + Can handle all errors (thrown by handlers _and_ when no dispatcher is found) + Flexible. Can check the code _and_ request data. - A bit big. The same effort like handling normal requests. * Handler outputs itself and signals that A handler wants to return a 404. It constructs the output string and signals the fact that there's no need for TBNL to build the HTML for the output. + Easy to implement(? clean?) + Flexible - Can't handle all errors. (No handler found? Then the code won't be executed.) Regards, Stefan

2 3

[tbnl-devel] Re: Socket leaks, again
by Edi Weitz 03 Aug '04

03 Aug '04

Hi Marc! I managed to delete your email I wanted to reply to but I'll try to copy the relevant parts from the c-l.net archive... :) > Heh?? Reporting and fixing problems is not really what I would call > making oneself a fool in public. ;-) Well, I wasn't so sure. I missed spectaclularly with my last one... :) > I think: Good catch! I looked at your patches, they look ok for me > so I commited them to the mod_lisp repository and switched my > servers to mod_lisp 2.35. OK, cool. Why, by the way, is it 2.35 and not 2.34? According to the mod_lisp website the latest release was 2.33 (which is also the one from Debian). > Please get the latest one (2.35) on the mod_lisp repository: > http://www.fractalconcept.com:8000/public/open-source/ I get no reply when I try to connect... :( Cheers, Edi.

1 2

[tbnl-devel] Tip: You can use <Files ...>, too
by Stefan Scholl 27 Jul '04

27 Jul '04

Hi! Looking for a right place for your "SetHandler lisp-handler"? It isn't restricted to <Location ...>. See http://httpd.apache.org/docs/sections.html Think about mod_lisp-pages with the extension .mlp: <Files *.mlp> SetHandler lisp-handler </Files> Path must exist. When you access http://some.server.test/foo/bar.mlp directory "foo" must exist. Regards, Stefan

3 2

[tbnl-devel] create-host-dispatch-proxy
by Stefan Scholl 27 Jul '04

27 Jul '04

Hi! (defun create-host-dispatch-proxy (host dispatcher) "Creates a proxy dispatcher which calls the given DISPATCHER when the HOST matches." (lambda (request) (when (and (host request) (string-equal host (host request))) (funcall dispatcher request)))) [Function] create-host-dispatch-proxy host dispatcher => dispatch-fn A convenience function which will return a dispatcher which calls the given dispatcher when the host matches. Purpose: Use the known functions for creating a dispatcher but add the feature to match the host first. Example: (create-host-dispatch-proxy "www.zappa.test" (create-static-file-dispatcher-and-handler "/tbnl/test/image.jpg" (make-pathname :name "fz" :type "jpg" :version nil :defaults (load-time-value *load-pathname*)) "image/jpeg")) (create-host-dispatch-proxy "www.king.test" (create-static-file-dispatcher-and-handler "/tbnl/test/image.jpg" (make-pathname :name "elvis" :type "jpg" :version nil :defaults (load-time-value *load-pathname*)) "image/jpeg")) This delivers fz.jpg on http://www.zappa.test/tbnl/test/image.jpg and elvis.jpg on http://www.king.test/tbnl/test/image.jpg Name has the word "proxy" in it. It's the right name, but it's no web-proxy as the user would expect. Other name for the function? The documentation uses the word "dispatcher" two times but one time it's the one created and the second time it's the one given in the lambda list. Hard to understand? Not easy to use in test/test.lisp. I don't know if this code should be included into TBNL. The list of convenience functions grows on and could become a bit complex. I see it more as an example on how to solve a certain problem when you want to run one lisp image with different (virtual) webserver addresses. But feel free to add it anyway. Regards, Stefan

2 1

[tbnl-devel] New version 0.2.6
by Edi Weitz 24 Jul '04

24 Jul '04

Hi! A new release is available from <http://weitz.de/files/tbnl.tgz>. Here's the relevant part from the changelog: Version 0.2.6 2004-07-24 Make CREATE-STATIC-FILE-DISPATCHER-AND-HANDLER thread-safe (caught by Jeff Caldwell) Added support for 'If-Modified-Since' request headers (provided by Stefan Scholl) Enjoy, Edi

1 2

[tbnl-devel] Patch for handling If-Modified-Since
by Stefan Scholl 24 Jul '04

24 Jul '04

Hi! A patch for handling If-Modified-Since is attached to this e-mail. Maybe it could be useful. I've only tested it with Mozilla 1.7.1 (with help from <http://livehttpheaders.mozdev.org/>). So additional tests would be fine. New function rfc1123-date for generating a date string for http headers. Function handle-if-modified-since wants a time and handles almost everything by itself. Checks against If-Modified-Since and returns a 304 return code if necessary. You have to set the Last-Modified header by yourself because you don't know if you can produce some output at this point. See create-static-file-dispatcher-and-handler in html.lisp for an example. I've improved create-static-file-dispatcher-and-handler by adding cache functionality with handle-if-modified-since and setting Last-Modified, and a simple check if the file exists. (==> 404 if not). You see that Last-Modified isn't set when the file doesn't exist. Regards, Stefan

2 5

[tbnl-devel] Question about dispatchers and their actions
by pete-tbnl-dev＠kazmier.com 24 Jul '04

24 Jul '04

I've got a question about the *dispatch-table* and the associated actions that are returned from the dispatch functions. Based on the code in modlisp.lisp: (loop for dispatcher in *dispatch-table* for action = (funcall dispatcher *request*) when action return (funcall action) finally (setf (return-code *reply*) +http-not-found+)))) Might it be desirable to check the return of (funcall action), and if nil, continue processing with the next dispatcher rather than returning and sending output at this point? If one truly wanted to return an empty body, one could still throw to the tbnl-handler-done tag which would bypass the rest of the dispatch table. I'm writing a template dispatcher which will process a request using HTML-TEMPLATE. Of course, this is only possible if I can map a request to a template (I may store the templates on the filesystem or in a db, not sure yet). If a template cannot be found for the request, then I'd like to fall back to processing the request with the rest of the dispatch table (which may just serve a static file or may be another template dispatcher thats resolves templates differently). In order to accomplish the above in the today's TBNL, I need to make the template resolution occur in the dispatch function (instead of the action function that the dispatcher returns) because I can return nil from the dispatcher if resolution fails and thus instruct TBNL to move on to the next dispatcher. In addition, I'd also like to make my dispatch decision based on a prefix. So I'd like to use create-prefix-dispatcher to limit the template processing to only part of the URI namespace. Thus, I need two 'tests' to occur in my dispatch function: 1) is there a prefix match (replicating the create-prefix-dispatcher functionality), and 2) does the request map to a template. With that said, I have created: (defun create-template-dispatcher (prefix resolver-function) #'(lambda (request) (when (prefix-match-p prefix (script-name request)) (let ((template (funcall resolver-function request))) (when template #'(lambda () (funcall *default-template-handler* template))))))) prefix-match-p is refactored code from create-prefix-dispatcher, but is essentially the same, does the request start with the given prefix? If it does not, I return nil, and processing continues with the rest of the dispatch table. Assuming the prefix does match, I then go on to call the template resolution function (resolver-function) which returns a template as a string or nil if a template could not be found. Template resolution might query a db or just read a template from the filesystem. Finally, if a template if found, I process the template with the default template handler which points to: (defun template-handler (template) (let ((context (assemble-context))) (with-output-to-string (*default-template-output*) (fill-and-print-template template context)))) Here is a sample template resolver function that maps a request to a template on the filesystem (only works on systems with '/' as a separator, my lisp path manipulations skills are absent at the moment): (defun filesystem-resolver (base &optional (request *request*)) (contents-of-file (parse-namestring (concatenate 'string base (script-name request))))) And here is my sample dispatch table that pulls templates from multiple locations on the filesystem depending on the request URI: (setf *dispatch-table* (list (create-template-dispatcher "/tbnl/space1" #'(lambda (request) (filesystem-resolver "/tmp" request))) (create-template-dispatcher "/tbnl/space2" #'(lambda (request) (filesystem-resolver "/var/www/html" request))) #'default-dispatcher)) Back to the original question and my rationale for asking it, I think I'd prefer if I didn't have to put all of my predicates for testing whether or not the template handler should be run within the dispatch function as it just doesn't seem to be the right spot for this logic. However, I definitely want the template handler executed based on the prefix. If TBNL could continue processing the dispatch table upon the return of nil from the handler (action), my dispatch table would look like: (setq *dispatch-table* (nconc (mapcar (lambda (args) (apply #'create-prefix-dispatcher args)) '(("/tbnl/space1" template-dispatcher) ("/tbnl/space2" template-dispatcher))) (list #'default-dispatcher))) And the template-dispatcher would consult a *resolver-table* and invoke the appropriate resolution function for the request. The nice thing about this approach is that my *dispatch-table* is not cluttered with non-dispatch related stuff such as template resolution. Of course that all requires the ability for an action (the template-dispatcher above) to signify that it cannot handle the request by returning a nil, which could then indicate to TBNL that the next entry in the *dispatch-table* should be tried. Would this functionality appeal to anyone else? Sorry about the length of email, but I thought it would be useful to share what I'm doing with TBNL. This is all just for fun and non-profit and my pursuit of learning lisp so if I've said something stupid or more specifically am not doing things in the traditional "lisp way", please let me know. Thanks, Pete

4 3

[tbnl-devel] 1) Aborted connections 2) create-static-file-dispatcher-and-handler
by Jeff Caldwell 24 Jul '04

24 Jul '04

Hi guys, Edi, I responded to your email last week but, by accident, didn't cc you. I told Pete I had just started my first Debian stable -> unstable upgrade and that my world was torn apart. The upgrade was a bear for me but I finally have the pieces glued back in place, at least enough to do rough work. I now know more than I did about X, BlackBox, KDE, apt-get, dselect, and all that jazz. A couple of things. Edi, your aborted connection patch is a good thing. I don't remember if you noted that the aborted connections never happen until after ab completes its run. It seems to me that the there is one additional request sent to mod_lisp as each httpd process terminates. I used the Apache logging feature and did 'tail -f error_log' to watch what happens as ab runs. Everything was fine until after ab completed. Then and only then did the aborted connection logs start appearing. I didn't measure it exactly but it seemed the messages dribbled along as httpd's terminated after reaching their approx. 150 peak. I put in enough log messages to verify that with ab -n 2000 -c 200, I got about 2199 total messages, of which 199 ended with aborts. With ab -n 2000 -c 1, there were no aborts. With ab -n 2000 -c 2, there was 1 abort. I was unable to find any fields in the request_rec that indicated that the request was a "duplicate". I set flags in the .. from memory, what are the functions, lib_child_init and lib_child_exit? .. but the "extra" call comes before any call to lib_child_exit. The requests are all full ones, and TBNL gets the full request and generates a full response, even through more requests are made than are asked for with ab's -n parameter. I'll do more research, for example looking at other Apache module's source code, and google searches, or writing a custom ab, when I get a chance. Tonight I'm leaving for vacation. I'm taking my laptop but my time and access will be occasional only. I'll be back the second Tuesday after today. In create-static-file-dispatcher-and-handler, I'm pretty sure the make-array is thread unsafe where it sits. Aren't only special variables thread safe? The code as is establishes the closure at best when the dispatch table is constructed. With multiple threads, they each are sharing buf, which isn't good for several reasons. In CMUCL, I got lots of "bad address" errors in a function underneath read-sequence on the stream reading fz.jpg. The errors seemed to go away when I moved the let/make-array inside the handler. Sorry I don't have time for a real test or real patches but I hope to be on the road in an hour. It's great to see an active tbnl-devel. Jeff --- Stefan Scholl <stesch(a)no-spoon.de> wrote: ... > (defun create-static-file-dispatcher-and-handler > (uri path content-type) > "Creates a dispatch function which will dispatch > to a handler > function which emits the file PATH with CONTENT-TYPE > if the URI > matches the SCRIPT-NAME of the request." > (let ((buf (make-array 8192 :element-type > #-:tbnl-bivalent-streams 'character > #+:tbnl-bivalent-streams > '(unsigned-byte 8)))) > (let ((handler #'(lambda () > (setf (content-type) > content-type) > (with-output-to-string (str) > (with-open-file (file path > :direction :input > > #+:tbnl-bivalent-streams :element-type > > #+:tbnl-bivalent-streams '(unsigned-byte 8)) > (do ((pos (read-sequence > buf file) (read-sequence buf file))) > ((zerop pos)) > (write-sequence buf str > :end pos))))))) > #'(lambda (request) > (when (equal (script-name request) uri) > handler))))) ... __________________________________ Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! http://advision.webevents.yahoo.com/yahoo/votelifeengine/

2 1

[tbnl-devel] create-static-file-dispatcher-and-handler
by Stefan Scholl 20 Jul '04

20 Jul '04

Hi! A simple convenience function for serving static files with TBNL. Example: (create-static-file-dispatcher-and-handler "/css/main.css" #P"css/main.css" "text/css") Maybe someone finds a better name and documentation. :-) (defun create-static-file-dispatcher-and-handler (uri path content-type) "Creates a dispatch function which will dispatch to a handler function which emits the file PATH with CONTENT-TYPE if the URI matches the SCRIPT-NAME of the request." (let ((buf (make-array 8192 :element-type #-:tbnl-bivalent-streams 'character #+:tbnl-bivalent-streams '(unsigned-byte 8)))) (let ((handler #'(lambda () (setf (content-type) content-type) (with-output-to-string (str) (with-open-file (file path :direction :input #+:tbnl-bivalent-streams :element-type #+:tbnl-bivalent-streams '(unsigned-byte 8)) (do ((pos (read-sequence buf file) (read-sequence buf file))) ((zerop pos)) (write-sequence buf str :end pos))))))) #'(lambda (request) (when (equal (script-name request) uri) handler))))) Regards, Stefan

2 1

[tbnl-devel] [PATCH] Typo in modlisp.lisp
by pete-tbnl-dev＠kazmier.com 20 Jul '04

20 Jul '04

This is certainly not worth an actual patch file ...

2 1