tbnl-devel

tbnl-devel@common-lisp.net

865 discussions

Start a nNew thread

[tbnl-devel] Tip: Checking for UTF-8 in submitted form
by Stefan Scholl 06 Aug '04

06 Aug '04

http://paste.lisp.org/display/2141

1 0

[tbnl-devel] Re: Socket leaks, again
by Edi Weitz 03 Aug '04

03 Aug '04

Hi Marc! I managed to delete your email I wanted to reply to but I'll try to copy the relevant parts from the c-l.net archive... :) > Heh?? Reporting and fixing problems is not really what I would call > making oneself a fool in public. ;-) Well, I wasn't so sure. I missed spectaclularly with my last one... :) > I think: Good catch! I looked at your patches, they look ok for me > so I commited them to the mod_lisp repository and switched my > servers to mod_lisp 2.35. OK, cool. Why, by the way, is it 2.35 and not 2.34? According to the mod_lisp website the latest release was 2.33 (which is also the one from Debian). > Please get the latest one (2.35) on the mod_lisp repository: > http://www.fractalconcept.com:8000/public/open-source/ I get no reply when I try to connect... :( Cheers, Edi.

1 2

[tbnl-devel] One more... :)
by Edi Weitz 02 Aug '04

02 Aug '04

Attached is another little patch. I made this a different patch from the previous one (2.37) because it might be a bit controversial. I've added two new headers - "apache-baseversion" which corresponds to the APACHE_BASEVERSION constant (something like "Apache/1.3.29") and "modlisp-version" which corresponds to the mod_lisp version string in mod_lisp.c. I think it'd be nice to have these so the Lisp side can prepare better diagnostic messages. One might argue that it's overhead to send these headers with every request. I personally think that this is not an issue because mod_lisp is already extremely fast and it doesn't really matter whether you send another 20 or 30 bytes through the socket or not. One might also argue that this is static information which should only be sent once. I'd disagree because - as we all know - you can update your Apache while your Lisp image keeps running. You can also update mod_lisp while your Lisp image keeps running. In both cases your Lisp application will have the new versions available immediately. Cheers, Edi.

2 2

[tbnl-devel] Tip: You can use <Files ...>, too
by Stefan Scholl 27 Jul '04

27 Jul '04

Hi! Looking for a right place for your "SetHandler lisp-handler"? It isn't restricted to <Location ...>. See http://httpd.apache.org/docs/sections.html Think about mod_lisp-pages with the extension .mlp: <Files *.mlp> SetHandler lisp-handler </Files> Path must exist. When you access http://some.server.test/foo/bar.mlp directory "foo" must exist. Regards, Stefan

3 2

[tbnl-devel] create-host-dispatch-proxy
by Stefan Scholl 27 Jul '04

27 Jul '04

Hi! (defun create-host-dispatch-proxy (host dispatcher) "Creates a proxy dispatcher which calls the given DISPATCHER when the HOST matches." (lambda (request) (when (and (host request) (string-equal host (host request))) (funcall dispatcher request)))) [Function] create-host-dispatch-proxy host dispatcher => dispatch-fn A convenience function which will return a dispatcher which calls the given dispatcher when the host matches. Purpose: Use the known functions for creating a dispatcher but add the feature to match the host first. Example: (create-host-dispatch-proxy "www.zappa.test" (create-static-file-dispatcher-and-handler "/tbnl/test/image.jpg" (make-pathname :name "fz" :type "jpg" :version nil :defaults (load-time-value *load-pathname*)) "image/jpeg")) (create-host-dispatch-proxy "www.king.test" (create-static-file-dispatcher-and-handler "/tbnl/test/image.jpg" (make-pathname :name "elvis" :type "jpg" :version nil :defaults (load-time-value *load-pathname*)) "image/jpeg")) This delivers fz.jpg on http://www.zappa.test/tbnl/test/image.jpg and elvis.jpg on http://www.king.test/tbnl/test/image.jpg Name has the word "proxy" in it. It's the right name, but it's no web-proxy as the user would expect. Other name for the function? The documentation uses the word "dispatcher" two times but one time it's the one created and the second time it's the one given in the lambda list. Hard to understand? Not easy to use in test/test.lisp. I don't know if this code should be included into TBNL. The list of convenience functions grows on and could become a bit complex. I see it more as an example on how to solve a certain problem when you want to run one lisp image with different (virtual) webserver addresses. But feel free to add it anyway. Regards, Stefan

2 1

[tbnl-devel] New version 0.2.6
by Edi Weitz 24 Jul '04

24 Jul '04

Hi! A new release is available from <http://weitz.de/files/tbnl.tgz>. Here's the relevant part from the changelog: Version 0.2.6 2004-07-24 Make CREATE-STATIC-FILE-DISPATCHER-AND-HANDLER thread-safe (caught by Jeff Caldwell) Added support for 'If-Modified-Since' request headers (provided by Stefan Scholl) Enjoy, Edi

1 2

[tbnl-devel] Patch for handling If-Modified-Since
by Stefan Scholl 24 Jul '04

24 Jul '04

Hi! A patch for handling If-Modified-Since is attached to this e-mail. Maybe it could be useful. I've only tested it with Mozilla 1.7.1 (with help from <http://livehttpheaders.mozdev.org/>). So additional tests would be fine. New function rfc1123-date for generating a date string for http headers. Function handle-if-modified-since wants a time and handles almost everything by itself. Checks against If-Modified-Since and returns a 304 return code if necessary. You have to set the Last-Modified header by yourself because you don't know if you can produce some output at this point. See create-static-file-dispatcher-and-handler in html.lisp for an example. I've improved create-static-file-dispatcher-and-handler by adding cache functionality with handle-if-modified-since and setting Last-Modified, and a simple check if the file exists. (==> 404 if not). You see that Last-Modified isn't set when the file doesn't exist. Regards, Stefan

2 5

[tbnl-devel] Question about dispatchers and their actions
by pete-tbnl-dev＠kazmier.com 24 Jul '04

24 Jul '04

I've got a question about the *dispatch-table* and the associated actions that are returned from the dispatch functions. Based on the code in modlisp.lisp: (loop for dispatcher in *dispatch-table* for action = (funcall dispatcher *request*) when action return (funcall action) finally (setf (return-code *reply*) +http-not-found+)))) Might it be desirable to check the return of (funcall action), and if nil, continue processing with the next dispatcher rather than returning and sending output at this point? If one truly wanted to return an empty body, one could still throw to the tbnl-handler-done tag which would bypass the rest of the dispatch table. I'm writing a template dispatcher which will process a request using HTML-TEMPLATE. Of course, this is only possible if I can map a request to a template (I may store the templates on the filesystem or in a db, not sure yet). If a template cannot be found for the request, then I'd like to fall back to processing the request with the rest of the dispatch table (which may just serve a static file or may be another template dispatcher thats resolves templates differently). In order to accomplish the above in the today's TBNL, I need to make the template resolution occur in the dispatch function (instead of the action function that the dispatcher returns) because I can return nil from the dispatcher if resolution fails and thus instruct TBNL to move on to the next dispatcher. In addition, I'd also like to make my dispatch decision based on a prefix. So I'd like to use create-prefix-dispatcher to limit the template processing to only part of the URI namespace. Thus, I need two 'tests' to occur in my dispatch function: 1) is there a prefix match (replicating the create-prefix-dispatcher functionality), and 2) does the request map to a template. With that said, I have created: (defun create-template-dispatcher (prefix resolver-function) #'(lambda (request) (when (prefix-match-p prefix (script-name request)) (let ((template (funcall resolver-function request))) (when template #'(lambda () (funcall *default-template-handler* template))))))) prefix-match-p is refactored code from create-prefix-dispatcher, but is essentially the same, does the request start with the given prefix? If it does not, I return nil, and processing continues with the rest of the dispatch table. Assuming the prefix does match, I then go on to call the template resolution function (resolver-function) which returns a template as a string or nil if a template could not be found. Template resolution might query a db or just read a template from the filesystem. Finally, if a template if found, I process the template with the default template handler which points to: (defun template-handler (template) (let ((context (assemble-context))) (with-output-to-string (*default-template-output*) (fill-and-print-template template context)))) Here is a sample template resolver function that maps a request to a template on the filesystem (only works on systems with '/' as a separator, my lisp path manipulations skills are absent at the moment): (defun filesystem-resolver (base &optional (request *request*)) (contents-of-file (parse-namestring (concatenate 'string base (script-name request))))) And here is my sample dispatch table that pulls templates from multiple locations on the filesystem depending on the request URI: (setf *dispatch-table* (list (create-template-dispatcher "/tbnl/space1" #'(lambda (request) (filesystem-resolver "/tmp" request))) (create-template-dispatcher "/tbnl/space2" #'(lambda (request) (filesystem-resolver "/var/www/html" request))) #'default-dispatcher)) Back to the original question and my rationale for asking it, I think I'd prefer if I didn't have to put all of my predicates for testing whether or not the template handler should be run within the dispatch function as it just doesn't seem to be the right spot for this logic. However, I definitely want the template handler executed based on the prefix. If TBNL could continue processing the dispatch table upon the return of nil from the handler (action), my dispatch table would look like: (setq *dispatch-table* (nconc (mapcar (lambda (args) (apply #'create-prefix-dispatcher args)) '(("/tbnl/space1" template-dispatcher) ("/tbnl/space2" template-dispatcher))) (list #'default-dispatcher))) And the template-dispatcher would consult a *resolver-table* and invoke the appropriate resolution function for the request. The nice thing about this approach is that my *dispatch-table* is not cluttered with non-dispatch related stuff such as template resolution. Of course that all requires the ability for an action (the template-dispatcher above) to signify that it cannot handle the request by returning a nil, which could then indicate to TBNL that the next entry in the *dispatch-table* should be tried. Would this functionality appeal to anyone else? Sorry about the length of email, but I thought it would be useful to share what I'm doing with TBNL. This is all just for fun and non-profit and my pursuit of learning lisp so if I've said something stupid or more specifically am not doing things in the traditional "lisp way", please let me know. Thanks, Pete

4 3

[tbnl-devel] 1) Aborted connections 2) create-static-file-dispatcher-and-handler
by Jeff Caldwell 24 Jul '04

24 Jul '04

Hi guys, Edi, I responded to your email last week but, by accident, didn't cc you. I told Pete I had just started my first Debian stable -> unstable upgrade and that my world was torn apart. The upgrade was a bear for me but I finally have the pieces glued back in place, at least enough to do rough work. I now know more than I did about X, BlackBox, KDE, apt-get, dselect, and all that jazz. A couple of things. Edi, your aborted connection patch is a good thing. I don't remember if you noted that the aborted connections never happen until after ab completes its run. It seems to me that the there is one additional request sent to mod_lisp as each httpd process terminates. I used the Apache logging feature and did 'tail -f error_log' to watch what happens as ab runs. Everything was fine until after ab completed. Then and only then did the aborted connection logs start appearing. I didn't measure it exactly but it seemed the messages dribbled along as httpd's terminated after reaching their approx. 150 peak. I put in enough log messages to verify that with ab -n 2000 -c 200, I got about 2199 total messages, of which 199 ended with aborts. With ab -n 2000 -c 1, there were no aborts. With ab -n 2000 -c 2, there was 1 abort. I was unable to find any fields in the request_rec that indicated that the request was a "duplicate". I set flags in the .. from memory, what are the functions, lib_child_init and lib_child_exit? .. but the "extra" call comes before any call to lib_child_exit. The requests are all full ones, and TBNL gets the full request and generates a full response, even through more requests are made than are asked for with ab's -n parameter. I'll do more research, for example looking at other Apache module's source code, and google searches, or writing a custom ab, when I get a chance. Tonight I'm leaving for vacation. I'm taking my laptop but my time and access will be occasional only. I'll be back the second Tuesday after today. In create-static-file-dispatcher-and-handler, I'm pretty sure the make-array is thread unsafe where it sits. Aren't only special variables thread safe? The code as is establishes the closure at best when the dispatch table is constructed. With multiple threads, they each are sharing buf, which isn't good for several reasons. In CMUCL, I got lots of "bad address" errors in a function underneath read-sequence on the stream reading fz.jpg. The errors seemed to go away when I moved the let/make-array inside the handler. Sorry I don't have time for a real test or real patches but I hope to be on the road in an hour. It's great to see an active tbnl-devel. Jeff --- Stefan Scholl <stesch(a)no-spoon.de> wrote: ... > (defun create-static-file-dispatcher-and-handler > (uri path content-type) > "Creates a dispatch function which will dispatch > to a handler > function which emits the file PATH with CONTENT-TYPE > if the URI > matches the SCRIPT-NAME of the request." > (let ((buf (make-array 8192 :element-type > #-:tbnl-bivalent-streams 'character > #+:tbnl-bivalent-streams > '(unsigned-byte 8)))) > (let ((handler #'(lambda () > (setf (content-type) > content-type) > (with-output-to-string (str) > (with-open-file (file path > :direction :input > > #+:tbnl-bivalent-streams :element-type > > #+:tbnl-bivalent-streams '(unsigned-byte 8)) > (do ((pos (read-sequence > buf file) (read-sequence buf file))) > ((zerop pos)) > (write-sequence buf str > :end pos))))))) > #'(lambda (request) > (when (equal (script-name request) uri) > handler))))) ... __________________________________ Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! http://advision.webevents.yahoo.com/yahoo/votelifeengine/

2 1

[tbnl-devel] create-static-file-dispatcher-and-handler
by Stefan Scholl 20 Jul '04

20 Jul '04

Hi! A simple convenience function for serving static files with TBNL. Example: (create-static-file-dispatcher-and-handler "/css/main.css" #P"css/main.css" "text/css") Maybe someone finds a better name and documentation. :-) (defun create-static-file-dispatcher-and-handler (uri path content-type) "Creates a dispatch function which will dispatch to a handler function which emits the file PATH with CONTENT-TYPE if the URI matches the SCRIPT-NAME of the request." (let ((buf (make-array 8192 :element-type #-:tbnl-bivalent-streams 'character #+:tbnl-bivalent-streams '(unsigned-byte 8)))) (let ((handler #'(lambda () (setf (content-type) content-type) (with-output-to-string (str) (with-open-file (file path :direction :input #+:tbnl-bivalent-streams :element-type #+:tbnl-bivalent-streams '(unsigned-byte 8)) (do ((pos (read-sequence buf file) (read-sequence buf file))) ((zerop pos)) (write-sequence buf str :end pos))))))) #'(lambda (request) (when (equal (script-name request) uri) handler))))) Regards, Stefan

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

tbnl-devel