June 2008 - tbnl-devel@common-lisp.net - mailman3.common-lisp.net

[hunchentoot-devel] googlebot revisitation rate excessive?
by Jeff Cunningham 05 Jul '08

05 Jul '08

Has anyone else noticed excessive visitation of pages served dynamically by Hunchentoot? On a couple of small sites virtually every page gets hit every five minutes by http://www.google.com/bot.html, regular as clockwork. Normally, I setup robots.txt to allow google to walk these sites but unless I can think of a way to tell them to back off to once a week or something I'm about to Disallow them. They're just little private sites and Google ends up being 99% of the traffic. It's silly. --Jeff

6 14

[hunchentoot-devel] cl-webdav-level-2.asd?
by Cyrus Harmon 30 Jun '08

30 Jun '08

Edi et al., It seems like it would be pretty straightforward to move the cl-webdav- level-2 stuff into it's own asdf system. There are a couple of things from my patch that should probably go into cl-webdav (and ht) as is, but most of it could easily moved to another package and still get the functionality we want. The only tricky bits are the the handlers. How should we handle the bits that need to reach down into the existing propfind/get handlers? We could just redefine these with the copy/pasted/modified code in the new asdf system, but that seems like a hacky OAOO violation that is likely to lead to subtle bugs down the road. Some sort of CLOS hackery with around methods? A list of functions to call for each handler? Thoughts? thanks, Cyrus

2 4

[hunchentoot-devel] chunked encoding for input
by Cyrus Harmon 28 Jun '08

28 Jun '08

The following patch fixes the case where one sends chunked data to a hunchentoot server with, say, a PUT request. We weren't paying attention to the transfer-encoding in get-post-data. Also, we should support looking for both "chunked" and "Chunked", so I use string-equal instead of equalp here. thanks, Cyrus

2 3

[hunchentoot-devel] cl-webdav level 2 support patch
by Cyrus Harmon 27 Jun '08

27 Jun '08

The attached patch provides initial support for DAV level 2, including locking and unlocking of files and enables cl-webdav servers to be used by the MacOS WebDAVFS client. Code review/comments greatly appreciated. thanks, Cyrus

2 3

[hunchentoot-devel] No timestamp in access log?
by Jeff Cunningham 26 Jun '08

26 Jun '08

This may have come up already and I missed it, but after upgrading to the new ediware version of hunchentoot I've noticed that my access logs no longer have timestamps, whereas the error logs still do. Cross-correlating them is, needless to say, a challenge. Looking at the code it doesn't appear to have the capability without modification. What is the thought here? --Jeff

2 3

[hunchentoot-devel] HT-AJAX website down
by Nimalan Mahendran 26 Jun '08

26 Jun '08

Hello, I'm trying to access the HT-AJAX website, but it seems to be inaccessible from either the cliki link or the hunchentoot link: http://www.cliki.net/HT-AJAX (http://pen.two-bytes.com/misc/ht-ajax.html) http://www.weitz.de/hunchentoot/#example (http://85.65.214.241/misc/ht-ajax.html) I can ping pen.two-bytes.com and it resolves to 85.65.214.241, but can't access the web server. Does anyone know whether the website has been down for a long while, whether HT-AJAX is still maintained and if there is an alternate location to get a copy of the library and documentation? Thanks! -- Nimalan Mahendran

3 3

[hunchentoot-devel] cl-webdav errors
by Cyrus Harmon 25 Jun '08

25 Jun '08

I'm getting some errors attempting to build the latest cl-webdav with the latest ediware/sbcl combo. 1) flex:flexi-stream-encoding-error is undefined. this is fixed by this: diff --git a/util.lisp b/util.lisp index 85b3afd..4f85165 100755 --- a/util.lisp +++ b/util.lisp @@ -90,5 +90,5 @@ then uses LATIN-1 if that fails." ;; LATIN-1... (handler-case (url-decode string +utf-8+) - (flex:flexi-stream-encoding-error () + (flex:external-format-encoding-error () (url-decode string +latin-1+)))) 2) perhaps the recent constant dumping stuff in SBCL is giving cl- webdav fits here, but I get the following error message: ; compiling file "/Users/sly/src/lisp/cl-webdav/cl-webdav-0.1.2-git/ properties.lisp" (written 17 APR 2007 12:42:08 AM): ; compiling (IN-PACKAGE :CL-WEBDAV) ; compiling (DEFUN PROPERTY-EQUAL ...) ; compiling (DEFUN DAV-PROPERTY-FUNCTION ...) ; file: /Users/sly/src/lisp/cl-webdav/cl-webdav-0.1.2-git/ properties.lisp ; in: DEFUN DAV-PROPERTY-FUNCTION ; (ASSOC CL-WEBDAV::PROPERTY CL-WEBDAV::+DAV-PROPERTY-ALIST+ :TEST #'STRING=) ; --> LET IF IF IF IF IF IF IF IF IF IF LET ; ==> ; '("lockdiscovery" . #<CLOSURE # {13A90B0D}>) ; ; caught ERROR: ; Objects of type FUNCTION can't be dumped into fasl files. ; --> LET IF IF IF IF IF IF IF IF IF IF ; ==> ; '("lockdiscovery" . #<CLOSURE # {13A90B0D}>) ; ; caught ERROR: ; Objects of type FUNCTION can't be dumped into fasl files. ; --> LET IF IF IF IF IF IF IF IF IF IF IF LET ; ==> ; '("supportedlock" . #<CLOSURE # {13A90B1D}>) ; ; caught ERROR: ; Objects of type FUNCTION can't be dumped into fasl files. ; --> LET IF IF IF IF IF IF IF IF IF IF IF ; ==> ; '("supportedlock" . #<CLOSURE # {13A90B1D}>) ; ; caught ERROR: ; Objects of type FUNCTION can't be dumped into fasl files. ; compiling (DEFUN GET-DEAD-PROPERTY ...) are we attempting to do something out of the CL spec here or is SBCL choking on legal code? thanks, Cyrus

2 7

[hunchentoot-devel] script-name*
by Cyrus Harmon 25 Jun '08

25 Jun '08

Well, it looks like Hans found the same problem I found with script- name, but, and this is a very minor stylistic point, since we already have the request here, why don't we just call (script-name request) instead of (script-name*)? thanks, Cyrus Index: headers.lisp =================================================================== --- headers.lisp (revision 3320) +++ headers.lisp (working copy) @@ -162,10 +162,10 @@ "The server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials \(e.g., bad password), or your browser doesn't understand how to supply the credentials required.") ((#.+http-forbidden+) (format nil "You don't have permission to access ~A on this server." - (script-name*))) + (script-name request))) ((#.+http-not-found+) (format nil "The requested URL ~A was not found on this server." - (script-name*))) + (script-name request))) ((#.+http-bad-request+) "Your browser sent a request that this server could not understand.") (otherwise "")) instead of

2 1

[hunchentoot-devel] Static file handling performance
by Andrei Stebakov 20 Jun '08

20 Jun '08

I had to come up with some way to cache dynamic files that I have to serve, so I ended up with a bunch of static files which I served by a simple function: (defun file-to-string (path) "Reads a file into a string" (if (probe-file path) (with-open-file (in path) (let ((str (make-string (file-length in)))) (read-sequence str in) str)))) The performance was very good, but then I thought that it's not the proper way to serve static files as there is a hunchentoot function handle-static-file. When I started using the hunchentoot's function the response time almost tripled and when I run "top" program to monitor CPU usage it jumps up to 60% (on my PIII 600 MHz) CPU, whereas using file-to-string CPU usage stays with 2% (maybe because the serving time is much shorter top doesn't catch that CPU peak). My question is what could be the reason I see this behaviour? (I am using SBCL 1.0.15 with latest dependences of hunchentoot-0.15.7) Thank you, Andrew

2 8

[hunchentoot-devel] Clozure CL HUNCHENTOOT-TEST Error
by Wade Humeniuk 16 Jun '08

16 Jun '08

I am using Clozure CL 1.2 rc1.2 with Hunchentoot. The hunchentoot-test pages that come with the install are not working completely. I had the to modify the test code in test.lisp to get a backtrace of what was happening. stream-direct-utf-8 is throwing an error and no content is coming back. (defvar *htest-debug-stream* nil) (defun stream-direct-utf-8 () (handler-bind ((error (lambda (c) (describe c *htest-debug-stream*) (format *htest-debug-stream* "~A" (hunchentoot:get-backtrace c))))) (setf (content-type) "text/html; charset=utf-8") (let ((stream (send-headers))) (setf (flex:flexi-stream-external-format stream) *utf-8*) (with-open-file (in (merge-pathnames "UTF-8-demo.html" *this-file*) :element-type 'flex:octet) (setq in (flex:make-flexi-stream in :external-format *utf-8*)) (loop for line = (read-line in nil nil) while line do (write-line line stream)))))) and modifying chunga::write-chunk to.. (defmethod write-chunk ((stream chunked-output-stream) sequence &key (start 0) (end (length sequence))) "Writes the contents of SEQUENCE from START to END to the underlying stream of STREAM as one chunk." (let ((output-stream (chunked-stream-output-stream stream))) (describe output-stream hunchentoot-test::*htest-debug-stream*) ;; chunk size (format output-stream "~X" (- end start)) ;; CRLF (write-string #.(format nil "~C~C" #\Return #\Linefeed) output-stream) ;; data #+:clisp (setf (flexi-stream-element-type output-stream) 'octet) (write-sequence sequence output-stream :start start :end end) #+:clisp (setf (flexi-stream-element-type output-stream) 'character) ;; CRLF (write-string #.(format nil "~C~C" #\Return #\Linefeed) output-stream))) Visting the page /hunchentoot/test/utf8-character.txt produces the output below. As far as I can tell from looking at the code there should be no instance of EXTERNAL-FORMAT since there is no method FLEXI-STREAMS::ENCODING-FACTOR for it (only subclasses of EXTERNAL-FORMAT). I have pretty well banged my head looking at the code in flexi-streams and chunga trying to determine the problem. This works fine for SBCL, and I see no conditional code for CCL that might explain the problem. Wade CL-USER> (hunchentoot:start-server :port 4242) #<HUNCHENTOOT::SERVER #x3000426BF81D> CL-USER> (setf hunchentoot-test::*htest-debug-stream* *standard-output*) #<SWANK-BACKEND::SLIME-OUTPUT-STREAM #x300040E2FC1D> #<SIMPLE-ERROR #x30004274A50D> Class: #<STANDARD-CLASS SIMPLE-ERROR> Wrapper: #<CLASS-WRAPPER #x3000404AC86D> Instance slots CCL::FORMAT-CONTROL: "No applicable method for args:~% ~s~% to ~s" CCL::FORMAT-ARGUMENTS: ((#<EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x30004122DA1D>) #<STANDARD-GENERIC-FUNCTION FLEXI-STREAMS::ENCODING-FACTOR #x300040FACA3F>) BACKTRACE--->>>>>> (7FC8F700D0B8) : 0 (GET-BACKTRACE #<SIMPLE-ERROR #x30004274A50D>) 285 (7FC8F700D120) : 1 (FUNCALL #'#<(:INTERNAL HUNCHENTOOT-TEST::STREAM-DIRECT-UTF-8)> #<SIMPLE-ERROR #x30004274A50D>) 173 (7FC8F700D140) : 2 (SIGNAL #<SIMPLE-ERROR #x30004274A50D> [...]) 997 (7FC8F700D198) : 3 (%ERROR #<SIMPLE-ERROR #x30004274A50D> NIL 17562639276619) 117 (7FC8F700D1C0) : 4 (CERROR "Try calling it again" "No applicable method for args:~% ~s~% to ~s" [...]) 933 (7FC8F700D258) : 5 (FUNCALL #'#<#<CCL::STANDARD-KERNEL-METHOD NO-APPLICABLE-METHOD (T)>> #<STANDARD-GENERIC-FUNCTION FLEXI-STREAMS::ENCODING-FACTOR #x300040FACA3F> [...]) 117 (7FC8F700D298) : 7 (FUNCALL #'#<#<STANDARD-METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T)>> #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004272511D> ".... (7FC8F700D300) : 8 (%%STANDARD-COMBINED-METHOD-DCODE '(# NIL #<#>) 17562639276650) 269 (7FC8F700D378) : 10 (WRITE-STRING "... (7FC8F700D3D8) : 11 (%PR-INTEGER 0 [...]) 965 (7FC8F700D478) : 12 (FORMAT-PRINT-NUMBER #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004272511D> 8187 16 NIL NIL NIL) 1829 (7FC8F700D4F8) : 13 (FUNCALL #'#<Anonymous Function #x3000409324EF> #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004272511D> NIL NIL [...]) 165 (7FC8F700D530) : 14 (SUB-FORMAT #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004272511D> 1 2) 2469 (7FC8F700D5E8) : 15 (DO-SUB-FORMAT #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004272511D>) 157 (7FC8F700D608) : 16 (FORMAT #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004272511D> "~X" [...]) 1725 (7FC8F700D728) : 17 (FUNCALL #'#<#<STANDARD-METHOD CHUNGA::WRITE-CHUNK (CHUNGA:CHUNKED-OUTPUT-STREAM T)>> #<CHUNGA:CHUNKED-IO-STREAM #x30004273F56D> #(60 33 68 79 67 ...) [...]) 237 (7FC8F700D770) : 18 (%%CHECK-KEYWORDS #(2 #(:END :START) #<METHOD-FUNCTION CHUNGA::WRITE-CHUNK #>) 17562639276793) 245 (7FC8F700D7F0) : 20 (FUNCALL #'#<#<STANDARD-METHOD CHUNGA::FLUSH-BUFFER (CHUNGA:CHUNKED-OUTPUT-STREAM)>> #<CHUNGA:CHUNKED-IO-STREAM #x30004273F56D>) 213 (7FC8F700D810) : 21 (FUNCALL #'#<#<STANDARD-METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (CHUNGA:CHUNKED-OUTPUT-STREAM T T T)>> #<CHUNGA:CHUNKED-IO-STREAM #x30004273F56D> #(184 177 224 184 162 ...) 0 63 [...]) 709 (7FC8F700D868) : 22 (WRITE-SEQUENCE #(184 177 224 184 162 ...) #<CHUNGA:CHUNKED-IO-STREAM #x30004273F56D> [...]) 765 (7FC8F700D8D8) : 23 (FUNCALL #'#<(:INTERNAL FLEXI-STREAMS::FLUSH-BUFFER (TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T)))>) 85 (7FC8F700D900) : 24 (FUNCALL #'#<(:INTERNAL FLEXI-STREAMS::WRITE-OCTET (TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T)))> 154) 69 (7FC8F700D938) : 25 (FUNCALL #'#<#<STANDARD-METHOD FLEXI-STREAMS::CHAR-TO-OCTETS (FLEXI-STREAMS::FLEXI-UTF-8-FORMAT T T)>> #<FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) #x300042642DBD> #\U+0E1A #<COMPILED-LEXICAL-CLOSURE (:INTERNAL FLEXI-STREAMS::WRITE-OCTET #) #x30004274A8CF>) 1629 (7FC8F700D970) : 26 (FUNCALL #'#<#<STANDARD-METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T)>> #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004273CBCD> " (7FC8F700D9F8) : 27 (%%STANDARD-COMBINED-METHOD-DCODE '(# NIL #<#>) 17562639276873) 269 (7FC8F700DA70) : 29 (WRITE-STRING "... (7FC8F700DAD0) : 30 (WRITE-LINE " (7FC8F700DB20) : 31 (STREAM-DIRECT-UTF-8) 805 (7FC8F700DBB0) : 32 (PROCESS-REQUEST '(# # # # # ...) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004273CBCD> :GET "/hunchentoot/test/utf8-character.txt" :HTTP/1.1) 3045 (7FC8F700DD18) : 33 (PROCESS-CONNECTION #<HUNCHENTOOT::SERVER #x3000426BF81D> #<BASIC-TCP-STREAM ISO-8859-1 (SOCKET/6) #x30004271F65D>) 5429 (7FC8F700DEB8) : 34 (RUN-PROCESS-INITIAL-FORM #<PROCESS hunchentoot-worker-1(13) [Active] #x30004271E9BD> '(#)) 717 (7FC8F700DF48) : 35 (FUNCALL #'#<(:INTERNAL CCL::%PROCESS-PRESET-INTERNAL)> #<PROCESS hunchentoot-worker-1(13) [Active] #x30004271E9BD> '(#)) 397 (7FC8F700DF98) : 36 (FUNCALL #'#<(:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION)>) 293 CHUNGA INFO--->>>>>>>>> Class: #<STANDARD-CLASS FLEXI-STREAMS:FLEXI-IO-STREAM> Wrapper: #<CLASS-WRAPPER #x300040FCCC3D> Instance slots CCL::SHARED-RESOURCE: NIL STREAM: #<BASIC-TCP-STREAM ISO-8859-1 (SOCKET/6) #x3000426A322D> FLEXI-STREAMS::EXTERNAL-FORMAT: #<EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x30004122DA1D> FLEXI-STREAMS::ELEMENT-TYPE: FLEXI-STREAMS::CHAR* FLEXI-STREAMS::COLUMN: 0 FLEXI-STREAMS::LAST-CHAR-CODE: NIL FLEXI-STREAMS::LAST-OCTET: NIL FLEXI-STREAMS::OCTET-STACK: NIL POSITION: 0 FLEXI-STREAMS::BOUND: NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x3000426A36BD> Class: #<STANDARD-CLASS FLEXI-STREAMS:FLEXI-IO-STREAM> Wrapper: #<CLASS-WRAPPER #x300040FCCC3D> Instance slots CCL::SHARED-RESOURCE: NIL STREAM: #<BASIC-TCP-STREAM ISO-8859-1 (SOCKET/6) #x3000426A322D> FLEXI-STREAMS::EXTERNAL-FORMAT: #<EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x30004122DA1D> FLEXI-STREAMS::ELEMENT-TYPE: FLEXI-STREAMS::CHAR* FLEXI-STREAMS::COLUMN: 0 FLEXI-STREAMS::LAST-CHAR-CODE: NIL FLEXI-STREAMS::LAST-OCTET: NIL FLEXI-STREAMS::OCTET-STACK: NIL POSITION: 0 FLEXI-STREAMS::BOUND: NIL

1 1