December 2007 - drakma-devel@common-lisp.net

[drakma-devel] CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE
by Nico de Jager 15 Jan '08

15 Jan '08

Hi I have a fingerprint scanning application, that uploads fingerprint images using Drakma to a Hunchentoot server. Everything works great as long as I don't use a secure connection. With https, I get a CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE condition on the server, and "Timeout while writing 8192 bytes to #<COMM:SOCKET-STREAM >" on the client. Sometimes the uploads are successful with https, though. The server runs LispWorks 5.0.2 on Debian Etch. The client runs on LispWorks 5.0.2 on Windows XP SP2. Part of the application pulls in the trivial-ldap module which depends on cl+ssl - not sure if this can be an issue as cl+ssl is not needed on LispWorks for Drakma and Hunchentoot. I am sure I am doing something stupid. The handler that does the actual upload on the client (which also uses Hunchentoot) is: (define-easy-handler (upload-fp-image.action :uri "/hunchentoot/patient-study-db/upload-fp-image.action") () (let ((cookie-jar (make-instance 'drakma:cookie-jar :cookies (list (make-instance 'drakma:cookie :name (session-value :hs-name) :value (session-value :hs-id) :path (session-value :hs-path) :domain (session-value :hs-domain)))))) (handler-case (let ((html-result (drakma:http-request (session-value :upload-url) :method :post :parameters (list (cons "finger" (session-value :finger)) (cons "img-file" (pathname *scanned-file*)) (cons "hs-id" (session-value :hs-id))) :user-agent (session-value :user-agent) :cookie-jar cookie-jar))) (delete-file *scanned-file*) (with-html-output-to-string (*standard-output*) (str html-result))) (condition (con) (delete-file *scanned-file*) (show-condition-html con (session-value :back-caption) (session-value :back-url)))))) The Hunchentoot server log spits out: [2007-12-12 11:35:39 [ERROR]] While parsing multipart/form-data parameters: The condition #<CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE 200B000C> occurred [2007-12-12 11:35:39] 10.38.1.45 - "POST /hunchentoot/patient-study-db/identify-patient.action HTTP/1.1" 200 1058 "-" "Opera/9.10 (Windows NT 5.1; U; en)" [2007-12-12 11:35:39 [ERROR]] Error while processing connection: Not all chunks from #<CHUNGA:CHUNKED-IO-STREAM 223A7893> have been read completely. Backtraces on the server and client are included below. Thanks. Nico Backtrace on server: Error: The condition #<CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE 223AB36C> occurred 1 (abort) Quit process. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 1 : 1 > :b Call to SIGNAL Call to (METHOD CHUNGA::FILL-BUFFER (CHUNGA:CHUNKED-INPUT-STREAM)) Call to (METHOD STREAM:STREAM-READ-BYTE (CHUNGA:CHUNKED-INPUT-STREAM)) Call to READ-BYTE Call to (METHOD FLEXI-STREAMS::READ-BYTE* (FLEXI-STREAMS::FLEXI-BINARY-INPUT-STREAM)) Call to (METHOD STREAM:STREAM-READ-CHAR (FLEXI-STREAMS::FLEXI-LATIN-1-INPUT-STREAM)) Call to READ-CHAR Call to (SUBFUNCTION RFC2388::RUN RFC2388::READ-UNTIL-NEXT-BOUNDARY) Call to RFC2388::READ-UNTIL-NEXT-BOUNDARY Call to (METHOD RFC2388:PARSE-MIME (STREAM T)) Call to HUNCHENTOOT::PARSE-RFC2388-FORM-DATA Call to (METHOD INITIALIZE-INSTANCE :AFTER (HUNCHENTOOT::REQUEST)) Call to CLOS::METHOD-COMBINATION-TEMPLATE Call to CLOS::MAKE-INSTANCE-FROM-CLASS-1 Call to HUNCHENTOOT::PROCESS-REQUEST Call to HUNCHENTOOT::PROCESS-CONNECTION Call to (SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS-STACK) Backtrace on client: Error: Timeout while writing 8192 bytes to #<COMM:SOCKET-STREAM 2009C0E7>. 1 (abort) Quit process. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 1 : 1 > :b Call to SIGNAL Call to (METHOD STREAM:STREAM-FLUSH-BUFFER (STREAM:BUFFERED-STREAM)) Call to (METHOD STREAM:STREAM-WRITE-SEQUENCE (STREAM:BUFFERED-STREAM T T T)) Call to CLOS::METHOD-COMBINATION-TEMPLATE Call to WRITE-SEQUENCE Call to WRITE-SEQUENCE Call to (METHOD CHUNGA::WRITE-CHUNK (CHUNGA:CHUNKED-OUTPUT-STREAM T)) Call to (METHOD CHUNGA::FLUSH-BUFFER (CHUNGA:CHUNKED-OUTPUT-STREAM)) Call to (METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (CHUNGA:CHUNKED-OUTPUT-STREAM T T T)) Call to WRITE-SEQUENCE Call to WRITE-SEQUENCE Call to DRAKMA::SEND-CONTENT Call to DRAKMA::SEND-CONTENT Call to (SUBFUNCTION 1 DRAKMA::MAKE-FORM-DATA-FUNCTION) Call to (SUBFUNCTION (LABELS DRAKMA::FINISH-REQUEST) DRAKMA:HTTP-REQUEST) Call to DRAKMA:HTTP-REQUEST Interpreted call to PATIENT-STUDY-DB-CLIENT::UPLOAD-FP-IMAGE.ACTION Call to HUNCHENTOOT::PROCESS-REQUEST Call to HUNCHENTOOT::PROCESS-CONNECTION Call to (SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS-STACK)

2 11

[drakma-devel] Re: [patch] drakma, support for connection-timeout on SBCL
by Stanislaw Halik 30 Dec '07

30 Dec '07

I've updated the patch to support the IO-TIMEOUT keyword argument. It's not possible to non-intrusively support read- and write-timeouts separately to bring the API in congruency with LispWorks. The resolver error gets changed to SIMPLE-ERROR because it's not an ERROR, but a CONDITION (signalled with ERROR, though). I've stress-tested the patch, running 30 threads doing HTTP-REQUEST. My rationale for including the patch - it doesn't change the program logic much, and except for the function definition and socket connection there's no special-casing for SBCL. The timeout option is crucial for me: my typical usage is running concurrent threads accessing URLs posted in spam to make the senders think I'm a used accessing the page, in turn sending more spam, addresses of which land in my spamtrap. Some of the URLs point to broken httpds, leaving an open connection without sending any replies. This leads to unfinished requests using threads and hanging for days. If you feel like maintaing a piece of code for SBCL, by all means please do so. I think other SBCL users might benefit from it. -- /\ / Jabber ID :: sthalik(a)jabber.autocom.pl \ \/ Unix stuff :: http://tehran.lain.pl \/\ Yet Another RBL :: http://rbl.lain.pl

3 11

[drakma-devel] Re: [patch] drakma, support for connection-timeout on SBCL
by Edi Weitz 22 Dec '07

22 Dec '07

On Fri, 21 Dec 2007 13:58:40 +0100, Stanislaw Halik <sthalik(a)tehran.lain.pl> wrote: > I attach the patch for drakma to support the `connection-timeout' > argument for `http-request' on SBCL. Hope you like it. Hi Stanislaw, I'm copying my reply to the Drakma mailing list where I think this discussion should take place. Thank you for the patches you sent, but I think that this functionality should rather be in usocket and not in Drakma. If it's offered by usocket, then Drakma will use it. Have you considered sending a patch to the usocket maintainers? Thanks again, Edi.

1 0

[drakma-devel] question about encodings and content length calculation
by Peter Eddy 17 Dec '07

17 Dec '07

Hi, I'm trying resolve a problem I'm having trying to send utf-8 data and the calculation of the length of that data. If I make the following post: (defconstant +utf-8+ (make-external-format :utf-8 :eol-style :lf) (http-request "http://odeon:5984" :content-type "application/xml" :external-format-out +utf-8+ :content "e" :method :put) The resulting request looks like this: PUT / HTTP/1.1 Host: odeon:5984 User-Agent: Drakma/0.11.1 (OpenMCL Version 1.1-pre-070722 (DarwinX8664); Darwin; 9.1.0; http://weitz.de/drakma/) Accept: */* Connection: close Content-Type: application/xml Content-Length: 1 e Which is right. But if I replace the "e" content with "é" (e accent acute), the request looks like this: PUT / HTTP/1.1 Host: odeon:5984 User-Agent: Drakma/0.11.1 (OpenMCL Version 1.1-pre-070722 (DarwinX8664); Darwin; 9.1.0; http://weitz.de/drakma/) Accept: */* Connection: close Content-Type: application/xml Content-Length: 1 .. The hex values of the ".." are C3 A9, which is the utf-8 value for é. The problem is, of course, that the content length is really 2. I assume I'm doing something wrong, but I don't know what exactly that would be. Should I be calculating the length myself? thanks for any pointers or suggestions. - Peter

2 2