October 2006 - drakma-devel@common-lisp.net

[drakma-devel] Error when chunked transfer encoding is used on clisp
by Anton Vodonosov 26 Oct '06

26 Oct '06

Hi! I've decided to repost this messsage here, because the message I've posted to lispweb mailing list contains stack backtrace produced by slime. But, as it turned out, slime debugger doesn't work well with clisp. The problem is that the http-request function fails when it's used for server with chunker transfer encoding. For example, all the following calls fail: (http-request "http://www.phpsecurepages.com/test/test.php") (http-request "http://bcheck.scanit.be/bcheck/") (http-request "http://google.com") (http-request "http://weitz.de/drakma/") whereas following cals work fine: (http-request "http://zappa.com/favicon.ico") (http-request "http://www.jalat.com/blogs/lisp?id=3") (http-request "http://www.lispworks.com/") (http-request "http://meme.b9.com/login.html") All the failed calls use cunked transfer encoding; all successful - don't use chunked transfer encoding. The error is always the same: SYSTEM::STORE: #\< does not fit into #(0 0 0 0 0 0 0 0 0 0 ...), bad type See backtrace bellow. Also, consider an example: ;; in order to use the ;; chunked-stream-input-stream ;; function (in-package :chunga) (setq *stream* (drakma:http-request "http://weitz.de/drakma/" :want-stream t)) *stream* => #<FLEXI-IO-STREAM #x1A223C11> (flexi-streams:flexi-stream-stream *stream*) => #<CHUNKED-IO-STREAM #x1A221BBD> (chunked-stream-input-stream (flexi-streams:flexi-stream-stream *stream*)) => #<FLEXI-IO-STREAM #x1A223C4D> (flexi-streams:flexi-stream-stream (chunked-stream-input-stream (flexi-streams:flexi-stream-stream *stream*))) => #<IO UNBUFFERED SOCKET-STREAM (UNSIGNED-BYTE 8) weitz.de:80> As far as I understand, chunked streams expect underlying stream to be binary. But in this case, chunked input stream has flexi stream as underlying stream. ----------------------------------------- Backtrace for the error signaled by (http-request "http://weitz.de/drakma/") ----------------------------------------- The error itself: #<SIMPLE-TYPE-ERROR #x1A22990D>: standard object type: SIMPLE-TYPE-ERROR 0 [$DATUM]: #\< 1 [$EXPECTED-TYPE]: (UNSIGNED-BYTE 8) 2 [$FORMAT-CONTROL]: "~S: ~S does not fit into ~S, bad type " 3 [$FORMAT-ARGUMENTS]: (SYSTEM::STORE #\< #<ARRAY (UNSIGNED-BYTE 8) (3566) #x1A228AE5>) ------------------------------------------- Backtrace: ... <11> #<SYSTEM-FUNCTION INVOKE-DEBUGGER> 1 frame binding variables (~ = dynamically): | ~ SYSTEM::*PRIN-STREAM* <--> #<UNBOUND> frame binding variables (~ = dynamically): | ~ *PRINT-READABLY* <--> NIL frame binding variables (~ = dynamically): | ~ *PRINT-ESCAPE* <--> T - #<SIMPLE-TYPE-ERROR #x1A230399> - #(0 0 0 0 0 ...) <12> #<SYSTEM-FUNCTION SYSTEM::STORE> 3 - #\< - 0 - #(8 #<SYSTEM-FUNCTION SYSTEM::VECTOR-INIT> #<SYSTEM-FUNCTION SYSTEM::VECTOR-UPD> #<SYSTEM-FUNCTION SYSTEM::VECTOR-ENDTEST> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-INIT> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-UPD> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-ENDTEST> #<SYSTEM-FUNCTION AREF> #<SYSTEM-FUNCTION SYSTEM::STORE> #<SYSTEM-FUNCTION IDENTITY> #<SYSTEM-FUNCTION SYSTEM::VECTOR-LENGTH> #<COMPILED-FUNCTION #:|67 91 (MAPC #'(LAMBDA # #) '(1 2 4 ...))-5-1|> #<SYSTEM-FUNCTION AREF> #<SYSTEM-FUNCTION SYSTEM::STORE> #<SYSTEM-FUNCTION SYSTEM::VECTOR-INIT-START> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-INIT-END>) - 3566 - 0 - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D> - #(0 0 0 0 0 ...) <13> #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - (UNSIGNED-BYTE 8) - CHARACTER - #<COMPILED-FUNCTION READ-SEQUENCE> - 3566 - 0 - (:START 0 :END 3566) - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D> - #(0 0 0 0 0 ...) <14> #<COMPILED-FUNCTION READ-SEQUENCE> - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> - 3566 - #<COMPILED-FUNCTION CHUNGA::GET-CHUNK-SIZE> - #(#(#(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>) #<COMPILED-FUNCTION CHUNGA::ADD-EXTENSIONS>) frame binding variables (~ = dynamically): | ~ CHUNGA::*CURRENT-ERROR-FUNCTION* <--> #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1-1|> - #(#(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>) - #(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>) - #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1|> - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> <15> #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1|> <16> #<STANDARD-GENERIC-FUNCTION CHUNGA::FILL-BUFFER> - #<COMPILED-FUNCTION #:|145 156 (DEFMETHOD STREAM-READ-BYTE (#) ...)-10-1-1|> - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> <17> #<COMPILED-FUNCTION #:|145 156 (DEFMETHOD STREAM-READ-BYTE (#) ...)-10-1-1|> <18> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-BYTE> - NIL - NIL - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> <19> #<SYSTEM-FUNCTION READ-BYTE> - #<COMPILED-FUNCTION FLEXI-STREAMS::READ-BYTE*> - NIL - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <20> #<COMPILED-FUNCTION FLEXI-STREAMS::READ-BYTE*> - #<COMPILED-FUNCTION FLEXI-STREAMS::READ-CHAR-8-BIT> - #(0 1 2 3 4 5 ... 253 254 255) - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <21> #<COMPILED-FUNCTION FLEXI-STREAMS::READ-CHAR-8-BIT> - #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE> <22> #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE> - #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE> - :LF - #(#(NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99> :ISO-8859-1) - #(NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>) - #<COMPILED-FUNCTION #:|221 275 (DEFMETHOD STREAM-READ-CHAR (#) ...)-11-1-1|> - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <23> #<COMPILED-FUNCTION #:|221 275 (DEFMETHOD STREAM-READ-CHAR (#) ...)-11-1-1|> <24> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-CHAR> - NIL - 0 - #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1-1|> - 8192 - 0 - "^@^@^@^@^@..." - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <25> #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1-1|> - #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1-1|> - #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1|> - NIL - 8192 - 0 - "^@^@^@^@^@..." - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <26> #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1|> <27> #<STANDARD-GENERIC-FUNCTION TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE> - 8192 - 0 - #<COMPILED-FUNCTION #:|30 105 (PROGN (DEFMETHOD GRAY:STREAM-READ-BYTE-SEQUENCE # ...) (DEFMETHOD GRAY:STREAM-WRITE-BYTE-SEQUENCE # ...) ...)-11-3-1-1|> - (0 8192) - "^@^@^@^@^@..." - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <28> #<COMPILED-FUNCTION #:|30 105 (PROGN (DEFMETHOD GRAY:STREAM-READ-BYTE-SEQUENCE # ...) (DEFMETHOD GRAY:STREAM-WRITE-BYTE-SEQUENCE # ...) ...)-11-3-1-1|> <29> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-CHAR-SEQUENCE> - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> - "^@^@^@^@^@..." - 8192 - 0 - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> - "^@^@^@^@^@..." <30> #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - CHARACTER - CHARACTER - #<COMPILED-FUNCTION READ-SEQUENCE> - #<UNBOUND> - #<UNBOUND> - NIL - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> - "^@^@^@^@^@ <31> #<COMPILED-FUNCTION READ-SEQUENCE> - NIL - 0 - "" - "^@^@^@^@^@ - T - CHARACTER - NIL - #(NIL ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=iso-8859-1"))) - #<COMPILED-FUNCTION DRAKMA::READ-BODY> - #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99> - T - ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=iso-8859-1")) - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <32> #<COMPILED-FUNCTION DRAKMA::READ-BODY> - NIL - #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99> - NIL - ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=iso-8859-1")) - 200 - :HTTP/1.1 - #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST> - NIL - NIL <33> #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST> - #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST> - #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-WRITE-HEADER> - #(#(#(NIL #<URI http://weitz.de/drakma/> NIL :HTTP/1.1 :GET NIL NIL NIL 5 (:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> T) #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-WRITE-HTTP-LINE>) unwind-protect frame - #(#(NIL #<URI http://weitz.de/drakma/> NIL :HTTP/1.1 :GET NIL NIL NIL 5 (:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> T) frame binding variables (~ = dynamically): | ~ FLEXI-STREAMS:*DEFAULT-EOL-STYLE* <--> :LF - NIL - :LF compiled block frame for HTTP-REQUEST - NIL - #(NIL #<URI http://weitz.de/drakma/> NIL :HTTP/1.1 :GET NIL NIL NIL 5 (:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL) - #<COMPILED-FUNCTION HTTP-REQUEST> - NIL - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - T - NIL - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - NIL - NIL - "*/*" - :DRAKMA - NIL - #<UNBOUND> - NIL - NIL - "application/x-www-form-urlencoded" - NIL - NIL - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - NIL - "http://weitz.de/drakma/" <34> #<COMPILED-FUNCTION HTTP-REQUEST> - #<COMPILED-FUNCTION HTTP-REQUEST> EVAL frame for form (HTTP-REQUEST "http://weitz.de/drakma/") frame binding environments VAR_ENV <--> NIL FUN_ENV <--> NIL BLOCK_ENV <--> NIL GO_ENV <--> NIL DECL_ENV <--> ((DECLARATION OPTIMIZE DECLARATION)) <35> ... Regards, -Anton

2 2

[drakma-devel] New version 0.5.3 (Was: [Lispweb] Announcement: Drakma)
by Edi Weitz 11 Oct '06

11 Oct '06

On Wed, 11 Oct 2006 14:17:59 +0200, Magnus Henoch <mange(a)freemail.hu> wrote: > I tested it with CLISP (CVS version, slightly post-2.40). I used > the examples on the web page; it worked fine until the binary > example. In line 121 of request.lisp, READ-SEQUENCE for some reason > extracts characters from STREAM, which it then fails to put into the > byte array RESULT: > > SYSTEM::STORE: #\G does not fit into #(0 0 0 0 0 0 0 0 0 0 ...), bad type > [Condition of type SIMPLE-TYPE-ERROR] > > The TEXTP parameter to READ-BODY is NIL, and > (flexi-streams:flexi-stream-element-type stream) returns CHARACTER. Hmm, CLISP certainly tries hard to be different from the pack... :) Anyway, please try the new version 0.5.3 and see if it makes a difference. Thanks for the report, Edi.

1 0

[drakma-devel] New version 0.5.2
by Edi Weitz 08 Oct '06

08 Oct '06

ChangeLog: Version 0.5.2 2006-10-08 Adher to user-provided content length if FORM-DATA is true Download: http://weitz.de/files/drakma.tar.gz Cheers, Edi.

1 0

[drakma-devel] New version 0.5.1
by Edi Weitz 06 Oct '06

06 Oct '06

ChangeLog: Version 0.5.1 2006-10-07 Take Content-Encoding header into account (due to a bug report by Gregory Tod) Download: http://weitz.de/files/drakma.tar.gz Cheers, Edi.

1 0

[drakma-devel] New Chunga version 0.2.0
by Edi Weitz 06 Oct '06

06 Oct '06

ChangeLog: Version 0.2.0 2006-10-06 Only wrap inner stream with flexi stream if really needed Download: http://weitz.de/files/chunga.tar.gz Cheers, Edi.

1 0