- drakma-devel@lists.common-lisp.net - mailman3.common-lisp.net

Re: [drakma-devel] eof error
by Frank Schorr 03 Apr '08

03 Apr '08

> > Is that a local server? Are you sure it can cope with parallel > requests? Are the different threads using the same session? Could > that be the problem? > It is a public server in the www which has many simultaneous users. I believe my session handling is correct, but it is not heavily tested. What makes me confident is that my program is sometimes successful. I'm always doing the same requests (apart from the session information) with 3 threads and about 20 requests in total. The eofs occur at random. My program has done some 10.000 requests from this site when run from the ide in the past. Do you believe the server could generate the error ? Wouldn't drakma handle it then ? May be this is a way to kick out a robot ? (They are using captchas at some places (which are not accessed by me)). I'm also not aware of using destructive operations... > > I will try to find a simple test case. > > That'd be the best. > I noticed that this is not easy. It appears that I will need some more time for this. ______________________________________________________ Bis 50 MB Dateianhänge? Kein Problem! http://freemail.web.de/club/landingpage.htm/?mc=025556

3 2

Re: [drakma-devel] eof error
by Frank Schorr 03 Apr '08

03 Apr '08

> > On Thu, 03 Apr 2008 22:34:59 +0200, Frank Schorr <franks-muc(a)web.de> wrote: > > > This is on win32/5.0.2 > > LispWorks, I assume. Yes (version 5.0.2) > > > I receive an eof error when accessing a web page. Backtrace below. > > > > This happens occasionally, every second or third call to > > HTTP-REQUEST, without regular pattern, when the call is made from a > > muproc thread. My code works fine when called from the ide. > > Hmm, hard to say without a reproducible test case. Does it happen > only with specific URLs or with arbitrary ones? > My code tries only one, parallel threads access the same web page, but different URLs, because sessions are tracked. But it works fine from the ide thread. I will try to find a simple test case. > > I suspect that the proper initial bindings should be made when the > > thread is spawned, but I was not able to get this done. > > Which initial bindings are you thinking of? see http://www.mu.dk/cl-muproc#muproc-spawn used to set up and initialize special variables in the new thread. I thought that every thread might want its own variable for things like a current stream or something similar to be independent from other threads. It was only guessing. _________________________________________________________________________ In 5 Schritten zur eigenen Homepage. Jetzt Domain sichern und gestalten! Nur 3,99 EUR/Monat! http://www.maildomain.web.de/?mc=021114

2 1

[drakma-devel] eof error
by Frank Schorr 03 Apr '08

03 Apr '08

This is on win32/5.0.2 I receive an eof error when accessing a web page. Backtrace below. This happens occasionally, every second or third call to HTTP-REQUEST, without regular pattern, when the call is made from a muproc thread. My code works fine when called from the ide. I suspect that the proper initial bindings should be made when the thread is spawned, but I was not able to get this done. Any help would be very welcome. Frank Error: End of file while reading stream #<FLEXI-STREAMS::FLEXI-BINARY-LATIN-1-IO-STREAM 2393B4BF>. 1 (abort) Quit process. Type :b for backtrace, :c <option number> to proceed, or :? for other options GPTO 1 : 1 > :< Call to INVOKE-DEBUGGER GPTO 2 : 1 > :b Call to INVOKE-DEBUGGER Call to SIGNAL Call to CHUNGA:READ-LINE* Call to DRAKMA::READ-STATUS-LINE Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION (LABELS DRAKMA::FINISH-REQUEST) DRAKMA:HTTP-REQUEST) Call to DRAKMA:HTTP-REQUEST Call to GET-PAGE-2 Call to CLIENT-SUBMIT-FORM Call to LET* Interpreted call to #:REMOTE-FUNCTION223724 Call to UTIL::WORKER Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION 1 UTIL::MAKE-PROCESS) Call to CL-MUPROC::MUPROC-PROCESS-FUNCTION Call to (HARLEQUIN-COMMON-LISP:SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS-STACK) _______________________________________________________________ Schon gehört? Der neue WEB.DE MultiMessenger kann`s mit allen: http://www.produkte.web.de/messenger/?did=3016

3 2

[drakma-devel] CLISP problem with READ-SEQUENCE; patch
by Anton Vodonosov 23 Mar '08

23 Mar '08

Hello. When using HTTP-REQUEST for non text content types, we have following error in CLISP: SYSTEM::STORE: #\< does not fit into #(0 0 0 0 0 0 0 0 0 0 ...), bad type Backtrace: 0: INVOKE-DEBUGGER 1: SYSTEM::STORE 2: EXT:READ-CHAR-SEQUENCE 3: READ-SEQUENCE 4: DRAKMA::READ-BODY 5: DRAKMA::HTTP-REQUEST-FINISH-REQUEST 6: unwind-protect frame 7: compiled block frame for HTTP-REQUEST 8: HTTP-REQUEST This is because CLISP doesn't support READ-SEQUENCE on a stream having STREAM-ELEMENT-TYPE == 'CHARACTER. The patch attached fixes this problem by explicit change of stream element type in the DRAKMA::READ-BODY function. Test URLs: http://www.google.com/calendar/feeds/pm55j8kg30dnm54ib2if9fuocc@group.calen… http://www.lisperati.com/lisplogo_alien_128.png Best regards, -- Anton

3 7

[drakma-devel] Parsing quoted cookie values (bug?)
by Victor Kryukov 18 Mar '08

18 Mar '08

Hello, I believe the following behaviour is a bug: (drakma::parse-set-cookie "session=\"1,2,3\"; domain=example.com; path=/") generates an error: While parsing cookie header "session=\"1,2,3\"; domain=example.com; path=/": Read character #\,, but expected #\=. [Condition of type SIMPLE-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [TERMINATE-THREAD] Terminate this thread (#<THREAD "new-repl-thread" {1004560551}>) Backtrace: 0: (CHUNGA::SIGNAL-UNEXPECTED-CHARS #\, #\=) 1: (CHUNGA:ASSERT-CHAR #<SB-IMPL::STRING-INPUT-STREAM {1004562A51}> #\=) 2: (CHUNGA:READ-NAME-VALUE-PAIR #<SB-IMPL::STRING-INPUT-STREAM {1004562A51}>)[:EXTERNAL] 3: (DRAKMA::PARSE-SET-COOKIE "session=\"1,2,3\"; domain=example.com; path=/") According to RFC2965[1], av-pairs = av-pair *(";" av-pair) av-pair = attr ["=" value] ; optional value attr = token value = token | quoted-string value is either a token or a quoted string. According to RFC2616[2], quoted-string = ( <"> *(qdtext | quoted-pair ) <"> ) qdtext = <any TEXT except <">> quoted string can contain any text except \", including commas. I know a particular web-site (reddit.com) that sets cookies of that form, and Firefox parses them without any problem. The problem seem to be in read-name-value-pair from CHUNGA: DRAKMA> (with-input-from-string (in "session=\"1,2,3\"; domain=example.com; path=/") (read-name-value-pair in :cookie-syntax t)) ("session" . "\"1") Best Regards, Victor. [1] http://www.faqs.org/rfcs/rfc2965.html [2] http://www.faqs.org/rfcs/rfc2616.html -- http://macrodefinition.blogspot.com

2 4

[drakma-devel] misleading error message
by Daniel Janus 13 Feb '08

13 Feb '08

Hello list, I've just found out that the following call to HTTP-REQUEST: (http-request "http://some.secret/uri" :method :post :parameters '(("TextField"))) yields the following error (very confusing, especially if the HTTP-REQUEST is buried deep on the call stack and its arguments are not outright visible) against Drakma 0.11.3: Illegal argument to file-namestring: NIL. [Condition of type TYPE-ERROR] This is because the cdr of one of the parameters (here, the only one) is nil, so it gets catched by the last OR clause in the last COND clause in DRAKMA::MAKE-FORM-DATA-FUNCTION, which causes the spurious error. Would it be possible to reformulate the message so that it's more clear what happened? Best regards, -- Daniel Janus <daniel(a)sentivision.com>

2 4

[drakma-devel] Odd LW error not in SBCL
by Jonathon McKitrick 16 Jan '08

16 Jan '08

I'm running a drakma-based app that gives an error on LispWorks, but not on SBCL. Error: End of file while reading stream #<FLEXI-STREAMS::FLEXI-BINARY-LATIN-1-IO-STREAM 200ED20F>. This is after a POST request, and the same error never occurs with SBCL. Is there something else I should be setting when calling http-request in LW versus SBCL? Jonathon McKitrick -- 'Java is a fine language, for low values of fine.'

2 5

[drakma-devel] CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE
by Nico de Jager 15 Jan '08

15 Jan '08

Hi I have a fingerprint scanning application, that uploads fingerprint images using Drakma to a Hunchentoot server. Everything works great as long as I don't use a secure connection. With https, I get a CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE condition on the server, and "Timeout while writing 8192 bytes to #<COMM:SOCKET-STREAM >" on the client. Sometimes the uploads are successful with https, though. The server runs LispWorks 5.0.2 on Debian Etch. The client runs on LispWorks 5.0.2 on Windows XP SP2. Part of the application pulls in the trivial-ldap module which depends on cl+ssl - not sure if this can be an issue as cl+ssl is not needed on LispWorks for Drakma and Hunchentoot. I am sure I am doing something stupid. The handler that does the actual upload on the client (which also uses Hunchentoot) is: (define-easy-handler (upload-fp-image.action :uri "/hunchentoot/patient-study-db/upload-fp-image.action") () (let ((cookie-jar (make-instance 'drakma:cookie-jar :cookies (list (make-instance 'drakma:cookie :name (session-value :hs-name) :value (session-value :hs-id) :path (session-value :hs-path) :domain (session-value :hs-domain)))))) (handler-case (let ((html-result (drakma:http-request (session-value :upload-url) :method :post :parameters (list (cons "finger" (session-value :finger)) (cons "img-file" (pathname *scanned-file*)) (cons "hs-id" (session-value :hs-id))) :user-agent (session-value :user-agent) :cookie-jar cookie-jar))) (delete-file *scanned-file*) (with-html-output-to-string (*standard-output*) (str html-result))) (condition (con) (delete-file *scanned-file*) (show-condition-html con (session-value :back-caption) (session-value :back-url)))))) The Hunchentoot server log spits out: [2007-12-12 11:35:39 [ERROR]] While parsing multipart/form-data parameters: The condition #<CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE 200B000C> occurred [2007-12-12 11:35:39] 10.38.1.45 - "POST /hunchentoot/patient-study-db/identify-patient.action HTTP/1.1" 200 1058 "-" "Opera/9.10 (Windows NT 5.1; U; en)" [2007-12-12 11:35:39 [ERROR]] Error while processing connection: Not all chunks from #<CHUNGA:CHUNKED-IO-STREAM 223A7893> have been read completely. Backtraces on the server and client are included below. Thanks. Nico Backtrace on server: Error: The condition #<CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE 223AB36C> occurred 1 (abort) Quit process. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 1 : 1 > :b Call to SIGNAL Call to (METHOD CHUNGA::FILL-BUFFER (CHUNGA:CHUNKED-INPUT-STREAM)) Call to (METHOD STREAM:STREAM-READ-BYTE (CHUNGA:CHUNKED-INPUT-STREAM)) Call to READ-BYTE Call to (METHOD FLEXI-STREAMS::READ-BYTE* (FLEXI-STREAMS::FLEXI-BINARY-INPUT-STREAM)) Call to (METHOD STREAM:STREAM-READ-CHAR (FLEXI-STREAMS::FLEXI-LATIN-1-INPUT-STREAM)) Call to READ-CHAR Call to (SUBFUNCTION RFC2388::RUN RFC2388::READ-UNTIL-NEXT-BOUNDARY) Call to RFC2388::READ-UNTIL-NEXT-BOUNDARY Call to (METHOD RFC2388:PARSE-MIME (STREAM T)) Call to HUNCHENTOOT::PARSE-RFC2388-FORM-DATA Call to (METHOD INITIALIZE-INSTANCE :AFTER (HUNCHENTOOT::REQUEST)) Call to CLOS::METHOD-COMBINATION-TEMPLATE Call to CLOS::MAKE-INSTANCE-FROM-CLASS-1 Call to HUNCHENTOOT::PROCESS-REQUEST Call to HUNCHENTOOT::PROCESS-CONNECTION Call to (SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS-STACK) Backtrace on client: Error: Timeout while writing 8192 bytes to #<COMM:SOCKET-STREAM 2009C0E7>. 1 (abort) Quit process. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 1 : 1 > :b Call to SIGNAL Call to (METHOD STREAM:STREAM-FLUSH-BUFFER (STREAM:BUFFERED-STREAM)) Call to (METHOD STREAM:STREAM-WRITE-SEQUENCE (STREAM:BUFFERED-STREAM T T T)) Call to CLOS::METHOD-COMBINATION-TEMPLATE Call to WRITE-SEQUENCE Call to WRITE-SEQUENCE Call to (METHOD CHUNGA::WRITE-CHUNK (CHUNGA:CHUNKED-OUTPUT-STREAM T)) Call to (METHOD CHUNGA::FLUSH-BUFFER (CHUNGA:CHUNKED-OUTPUT-STREAM)) Call to (METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE (CHUNGA:CHUNKED-OUTPUT-STREAM T T T)) Call to WRITE-SEQUENCE Call to WRITE-SEQUENCE Call to DRAKMA::SEND-CONTENT Call to DRAKMA::SEND-CONTENT Call to (SUBFUNCTION 1 DRAKMA::MAKE-FORM-DATA-FUNCTION) Call to (SUBFUNCTION (LABELS DRAKMA::FINISH-REQUEST) DRAKMA:HTTP-REQUEST) Call to DRAKMA:HTTP-REQUEST Interpreted call to PATIENT-STUDY-DB-CLIENT::UPLOAD-FP-IMAGE.ACTION Call to HUNCHENTOOT::PROCESS-REQUEST Call to HUNCHENTOOT::PROCESS-CONNECTION Call to (SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS-STACK)

2 11

[drakma-devel] END-OF-FILE
by Victor Kryukov 14 Jan '08

14 Jan '08

Hello list, I've the following condition end of file on #<FLEXI-STREAMS::FLEXI-LATIN-1-IO-STREAM {10050A5171}> [Condition of type END-OF-FILE] while trying to do (drakma:http-request "http://www.bluishcoder.co.nz/atom.xml") Below is the relevant backtrace part: 0: (READ-CHAR #<FLEXI-STREAMS::FLEXI-LATIN-1-IO-STREAM {10050A5171}> T NIL #<unused argument>) 1: (CHUNGA:READ-LINE* #<FLEXI-STREAMS::FLEXI-LATIN-1-IO-STREAM {10050A5171}> NIL) 2: (DRAKMA::READ-STATUS-LINE #<FLEXI-STREAMS::FLEXI-LATIN-1-IO-STREAM {10050A5171}> NIL) 3: ((LABELS DRAKMA::FINISH-REQUEST) NIL NIL) 4: (DRAKMA:HTTP-REQUEST "http://www.bluishcoder.co.nz/atom.xml") I can access that URI without any problems using my browser. I'm slightly confused what's going on - could somebody please provide any comments? I'm using DRAKMA v. 0.11.1 and SBCL 1.0.13 on x86-64 linux (Ubuntu 7.10). Best Regards, Victor.

2 2

[drakma-devel] Re: [patch] drakma, support for connection-timeout on SBCL
by Stanislaw Halik 30 Dec '07

30 Dec '07

I've updated the patch to support the IO-TIMEOUT keyword argument. It's not possible to non-intrusively support read- and write-timeouts separately to bring the API in congruency with LispWorks. The resolver error gets changed to SIMPLE-ERROR because it's not an ERROR, but a CONDITION (signalled with ERROR, though). I've stress-tested the patch, running 30 threads doing HTTP-REQUEST. My rationale for including the patch - it doesn't change the program logic much, and except for the function definition and socket connection there's no special-casing for SBCL. The timeout option is crucial for me: my typical usage is running concurrent threads accessing URLs posted in spam to make the senders think I'm a used accessing the page, in turn sending more spam, addresses of which land in my spamtrap. Some of the URLs point to broken httpds, leaving an open connection without sending any replies. This leads to unfinished requests using threads and hanging for days. If you feel like maintaing a piece of code for SBCL, by all means please do so. I think other SBCL users might benefit from it. -- /\ / Jabber ID :: sthalik(a)jabber.autocom.pl \ \/ Unix stuff :: http://tehran.lain.pl \/\ Yet Another RBL :: http://rbl.lain.pl

3 11