January 2012 - tbnl-devel@common-lisp.net

[hunchentoot-devel] Ediware packages missing from Git
by Patrick May 12 Feb '12

12 Feb '12

Are there any plans to add puri (used by drakma) and trivial-gray-streams (used by chunga and flexi-streams) to the git repository so that all required packages are available in one spot? (Yes, I know that cxml and uffi, used by cl-webdav and cl-gd respectively, are also not in Git, but I'm primarily using Hunchentoot, cl-who, and Drakma.) Regards, Patrick

2 2

[hunchentoot-devel] GET parameters and request external format
by Dmitriy Ivanov 24 Jan '12

24 Jan '12

Hello folks, OT1H, the charset and external format of a request are extracted from the Content-Type header and used to percent-decode post parameters in the function maybe-read-post-parameters. OTOH, get parameters are always 1) initialized in the method #<initialize-instance :after (request) >, 2) percent-decoded using *hunchentoot-default-external-format*. That works fine when all pages of the site use a the same external format - just setting *hunchentoot-default-external-format* globally is enough. But if one wanted various pages to be in different formats, that could be inappropriate. For now, it is even buggy because within the current flexi-streams version, the decode error is not suppressed by following code: (let ((*substitution-char* #\?)) (form-url-encoded-list-to-alist (split "&" query-string))) I have bumped into all this while trying to process forms in the :windows-1251 external format. AFAIK, user agents percent-encode get parameters using in the charset specified in the previous server response. Without post parameters, user agents do not supply the charset in Content-Type as a rule. Thus only a Hunchentoot request handler can definitely know what the charset is actually. All that means that we should postpone retrieval of get parameters as far as possible. Maybe it is worth not to specify the :initform for the get-parameters slot and leave it unbound? Then we could initialize this slot with the following: (defmethod slot-unbound (class (request request) (slot-name (eql 'get-parameters))) (setf (slot-value request 'get-parameters) (form-url-encoded-list-to-alist (split "&" (query-string request)) (request-external-format request)))) The only question is how to compute such a request-external-format. Any takers? -- Sincerely, Dmitriy Ivanov lisp.ystok.ru

1 0

[hunchentoot-devel] Error on LWW 4.4.6: -1 cannot be converted to foreign type (:UNSIGNED-INTEGER-TYPE 32).
by Dmitriy Ivanov 19 Jan '12

19 Jan '12

Hello folks, I have bumped into the error while playing around: 1. (hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4242 :write-timeout nil)) ; for lispworks4 2. Browsing results from test-handlers.lisp. It worked fine until I made a break for half an hour. When I come back to my computer and click on some link, LispWorks signaled. The odd thing is that in call to COMM::LISTEN-AND-ATTACH-STREAM-FROM-SOCKET, COMM::SOCKET : 464 But in call to (METHOD HUNCHENTOOT:CREATE-REQUEST-HANDLER-THREAD (HUNCHENTOOT:ONE-THREAD-PER-CONNECTION-TASKMASTER T)) HUNCHENTOOT::SOCKET : -1 Looks like LispWorks (or even Windows) error. Has anybody observed something like this? The backtrace excerpt is below. --------------------------------------------------- Condition: -1 cannot be converted to foreign type (:UNSIGNED-INTEGER-TYPE 32). Call to (SUBFUNCTION 2 COMM:START-UP-SERVER) (offset 43) COMM::SOCKET : NIL CONDITION : #<FLI:FOREIGN-TYPE-ERROR 206AB0CC> Call to (SUBFUNCTION 1 COMM::LISTEN-AND-ATTACH-STREAM-NO-ERROR) (offset 41) CONDITION : #<FLI:FOREIGN-TYPE-ERROR 206AB0CC> Binding frame: CONDITIONS::*IN-SIGNAL-CATCH* : T Call to SIGNAL (offset 699) CONDITIONS::DATUM : #<FLI:FOREIGN-TYPE-ERROR 206AB0CC> CONDITIONS::ARGUMENTS : NIL Binding frame: CONDITIONS::*IN-SIGNAL-CATCH* : NIL Catch frame: CONDITIONS::SIGNAL-CATCH Binding frame: CONDITIONS::*BROKEN-ON-SIGNALS* : NIL Catch frame: #<unbound> Binding frame: FLI::*FOREIGN-OBJECT-SCOPE-STACK-DETAILS* : #(16 #1=(8519856 . 4096) (#1#)) Call to COMM::GETPEERNAME (offset 227) COMM::S : -1 COMM::NAME : #<Pointer to type (:STRUCT COMM::SOCKADDR) = #x008200B0> COMM::NAMELEN : 16 Catch frame: #<unbound> Binding frame: FLI::*FOREIGN-OBJECT-SCOPE-STACK-DETAILS* : #(0 NIL NIL) Call to COMM::GET-SOCKET-ADDRESS-AUX (offset 370) COMM::SOCKET : -1 FUNCTION : COMM::GETPEERNAME Call to HUNCHENTOOT::GET-PEER-ADDRESS-AND-PORT (offset 43) HUNCHENTOOT::SOCKET : -1 Call to (METHOD HUNCHENTOOT:CREATE-REQUEST-HANDLER-THREAD (HUNCHENTOOT:ONE-THREAD-PER-CONNECTION-TASKMASTER T)) (offset 97) HUNCHENTOOT:TASKMASTER : #<HUNCHENTOOT:ONE-THREAD-PER-CONNECTION-TASKMASTER 22087454> HUNCHENTOOT::SOCKET : -1 CLOS::.ISL. : :DONT-KNOW Catch frame: #<unbound> Call to COMM::LISTEN-AND-ATTACH-STREAM-FROM-SOCKET (offset 133) COMM::SOCKET : 464 FUNCTION : #<closure (SUBFUNCTION 3 (METHOD HUNCHENTOOT:START-LISTENING (HUNCHENTOOT:ACCEPTOR))) 22087F32> Catch frame: (NIL) Call to COMM::LISTEN-AND-ATTACH-STREAM (offset 619) FUNCTION : #<closure (SUBFUNCTION 3 (METHOD HUNCHENTOOT:START-LISTENING (HUNCHENTOOT:ACCEPTOR))) 22087F32> COMM::SERVICE : 4242 COMM::ANNOUNCE : #<closure (SUBFUNCTION 2 COMM:START-UP-SERVER) 22087F62> COMM::BACKLOG : 5 COMM::ADDRESS : NIL Catch frame: (NIL) Call to COMM::LISTEN-AND-ATTACH-STREAM-NO-ERROR (offset 284) FUNCTION : #<closure (SUBFUNCTION 3 (METHOD HUNCHENTOOT:START-LISTENING (HUNCHENTOOT:ACCEPTOR))) 22087F32> COMM::SERVICE : 4242 COMM::ANNOUNCE : #<closure (SUBFUNCTION 2 COMM:START-UP-SERVER) 22087F62> COMM::BACKLOG : 5 COMM::ADDRESS : NIL Catch frame: MP::PROCESS-TAG ... -- Sincerely, Dmitriy Ivanov lisp.ystok.ru

2 1

[hunchentoot-devel] What does +latin-1+ means ?
by z_axis 13 Jan '12

13 Jan '12

Can we use +utf-8+ instead of +latin-1+ in handler ? It is a newbie question but the following code doesnot work using +latin-1+: (let ((out (flexi-streams:make-flexi-stream (tbnl:send-headers) :external-format tbnl::+utf-8+))) ; tbnl::+;latin-1+ doesnot work (copy-stream in out 'character)) where, "in" holds utf-8 data. In hunchentoot-1.2.2, it says that +latin-1+ is "A FLEXI-STREAMS external format used for `faithful' input and output of binary data.", which i cannot understand very well. Sincerely!

1 0

[hunchentoot-devel] Why can't the python script get the post parameters from hunchentoot-cgi?
by z_axis 11 Jan '12

11 Jan '12

The hunchentoot-cgi basically works with hunchentoot-1.2.2 now with a few modification: 1. In #'handle-cgi-script, replacing :external-format tbnl::+latin-1+ with :external-format tbnl::+utf-8+ 2. As WhiteCat suggested, use #'make-pathname as below: (pushnew (hunchentoot-cgi::create-cgi-dispatcher-and-handler "/cgi-bin/" (make-pathname :directory '(:absolute "media" "E" "myapp" "cgi-bin"))) dispatch-table :test #'equal) However, I do not understand why the python script cannot get the POSTed parameter, i.e., when visiting http://127.0.0:8000/cgi-bin/login.py?cmd=view, login.py can get the parameter cmd's value sucessfully, but when posted by the following form, login.py cannot get all the posted values (no matter hidden or not): <html><body> <form method='POST' action='cgi-bin/login.py'> <input type='text' name='userid'> <input type='password' name='userpwd'> <input type='submit' value='Login'> <input type='hidden' name='cmd' value='view'> </form> </body></html> I guess hunchentoot-cgi does not pass the posted parameter(s) to the python script according to the environment: GET http://127.0.0.1:8000/cgi-bin/nav.py?userid=xyz&cmd=view (SERVER_SOFTWARE=hunchentoot/1.2.2 SERVER_NAME=127.0.0.1 GATEWAY_INTERFACE=CGI/1.1 SERVER_PROTOCOL=HTTP/1.1 SERVER_PORT=8000 REQUEST_METHOD=GET CONTENT_TYPE=text/html CONTENT_LENGTH=NIL SCRIPT_NAME=/cgi-bin/nav.py QUERY_STRING=userid=xyz&cmd=view REMOTE_ADDR=127.0.0.1 HTTP_HOST=NIL REQUEST_URI=/cgi-bin/nav.py?userid=xyz&cmd=view SERVER_ADDR=NIL HTTP_USER_AGENT=Mozilla/5.0 (X11; Linux i686; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 HTTP_REFERER=http://127.0.0.1:8000/) POST to /cgi-bin/nav.py (SERVER_SOFTWARE=hunchentoot/1.2.2 SERVER_NAME=127.0.0.1 GATEWAY_INTERFACE=CGI/1.1 SERVER_PROTOCOL=HTTP/1.1 SERVER_PORT=8000 REQUEST_METHOD=POST CONTENT_TYPE=text/html CONTENT_LENGTH=NIL POST_PARAMETERS=((userid . xyz) (userpwd . 123) (cmd . view)) SCRIPT_NAME=/cgi-bin/nav.py QUERY_STRING=NIL REMOTE_ADDR=127.0.0.1 HTTP_HOST=NIL REQUEST_URI=/cgi-bin/nav.py SERVER_ADDR=NIL HTTP_USER_AGENT=Mozilla/5.0 (X11; Linux i686; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 Please shed a light on me, thanks!

4 9

[hunchentoot-devel] Problem with (setf (hunchentoot:reply-external-format*) ...)
by Raymond Wiker 10 Jan '12

10 Jan '12

I have (finally!) updated my local installation of hunchentoot. Testing indicates that things work as expected, except for one thing: calling (setf (hunchentoot:reply-external-format*) ...) in a request handler causes the request to abort. This is probably not a big issue, since I probably don't need to set the external format (it appears to get the right value automatically). I'm reporting it here because it may actually be caused by a bug; alternatively, it could be that I'm doing something stupid, which would then be useful to know. The code snippet below works (after making an easy-acceptor). Removing the #+nil causes it to break. (hunchentoot:define-easy-handler (test :uri "/test") (foo) (setf (hunchentoot:content-type*) "text/plain") #+nil (setf (hunchentoot:reply-external-format*) :utf-8) (format nil "Host: ~a~%Proto: ~a (~a)~%Script: ~a~%requrl: ~a~%foo: ~a" (hunchentoot:host) (hunchentoot:server-protocol*) (type-of (hunchentoot:server-protocol*)) (hunchentoot:script-name*) (hunchentoot:request-uri*) foo))

2 5

[hunchentoot-devel] Problems with big uploads
by Stefan Nobis 09 Jan '12

09 Jan '12

Hi. I have written a small and very simple web-service to which I want to upload files. With the current version 1.2.1 of hunchentoot I have problems uploading big files, say 1GB. As far as I remeber, with older versions (1.0 and 1.1) big uploads worked well. Basically the following code is in use: --8<---------------cut here---------------start------------->8--- (hunchentoot:define-easy-handler (handle-upload :uri "/path/to/upload-service") () (let ((uploaded (when (and (boundp 'hunchentoot:*request*) (hunchentoot:post-parameter "filename")) (handle-file (hunchentoot:post-parameter "filename"))))) (generate-html-code))) --8<---------------cut here---------------end--------------->8--- And handle-file looks like this: --8<---------------cut here---------------start------------->8--- (defun handle-file (post-parameter) (ht-log :info "Handling file upload with params: '~A'." post-parameter) (when (and post-parameter (listp post-parameter)) (destructuring-bind (path filename content-type) post-parameter (declare (ignore content-type)) ;; strip directory info send by Windows browsers (when (search "Windows" (hunchentoot:user-agent) :test #'char-equal) (setf filename (ppcre:regex-replace ".*\\\\" filename ""))) (fad:copy-file path (ensure-directories-exist (merge-pathnames filename *unsecure-upload-dir*)) :overwrite t) filename))) --8<---------------cut here---------------end--------------->8--- It seems that hunchentoot tries to read the whole stream into memory and that the heap is too small (the server has only 1GB RAM and the heap of the sbcl process is limited to about 600MB). Is there any (easy) way to let hunchentoot load the data in small chunks in order to limit the maximal amount of memory used, independent of the file size? -- Until the next mail..., Stefan.

2 3