[drakma-devel] Error when chunked transfer encoding is used on clisp

Hi!

I've decided to repost this messsage here, because the message I've posted to lispweb mailing list contains stack backtrace produced by slime. But, as it turned out, slime debugger doesn't work well with clisp.

The problem is that the http-request function fails when it's used for server with chunker transfer encoding.

For example, all the following calls fail:

(http-request "http://www.phpsecurepages.com/test/test.php") (http-request "http://bcheck.scanit.be/bcheck/") (http-request "http://google.com") (http-request "http://weitz.de/drakma/")

whereas following cals work fine:

(http-request "http://zappa.com/favicon.ico") (http-request "http://www.jalat.com/blogs/lisp?id=3") (http-request "http://www.lispworks.com/") (http-request "http://meme.b9.com/login.html")

All the failed calls use cunked transfer encoding; all successful - don't use chunked transfer encoding.

The error is always the same: SYSTEM::STORE: #< does not fit into #(0 0 0 0 0 0 0 0 0 0 ...), bad type

See backtrace bellow.

Also, consider an example:

;; in order to use the ;; chunked-stream-input-stream ;; function (in-package :chunga)

(setq *stream* (drakma:http-request "http://weitz.de/drakma/" :want-stream t))

*stream* => #<FLEXI-IO-STREAM #x1A223C11>

(flexi-streams:flexi-stream-stream *stream*) => #<CHUNKED-IO-STREAM #x1A221BBD>

(chunked-stream-input-stream (flexi-streams:flexi-stream-stream *stream*)) => #<FLEXI-IO-STREAM #x1A223C4D>

(flexi-streams:flexi-stream-stream (chunked-stream-input-stream (flexi-streams:flexi-stream-stream *stream*))) => #<IO UNBUFFERED SOCKET-STREAM (UNSIGNED-BYTE 8) weitz.de:80>

As far as I understand, chunked streams expect underlying stream to be binary.

But in this case, chunked input stream has flexi stream as underlying stream.

----------------------------------------- Backtrace for the error signaled by (http-request "http://weitz.de/drakma/") ----------------------------------------- The error itself:

#<SIMPLE-TYPE-ERROR #x1A22990D>: standard object type: SIMPLE-TYPE-ERROR 0 [$DATUM]: #< 1 [$EXPECTED-TYPE]: (UNSIGNED-BYTE 8) 2 [$FORMAT-CONTROL]: "~S: ~S does not fit into ~S, bad type " 3 [$FORMAT-ARGUMENTS]: (SYSTEM::STORE #< #<ARRAY (UNSIGNED-BYTE 8) (3566) #x1A228AE5>)

------------------------------------------- Backtrace:

... <11> #<SYSTEM-FUNCTION INVOKE-DEBUGGER> 1 frame binding variables (~ = dynamically): | ~ SYSTEM::*PRIN-STREAM* <--> #<UNBOUND> frame binding variables (~ = dynamically): | ~ *PRINT-READABLY* <--> NIL frame binding variables (~ = dynamically): | ~ *PRINT-ESCAPE* <--> T - #<SIMPLE-TYPE-ERROR #x1A230399> - #(0 0 0 0 0 ...) <12> #<SYSTEM-FUNCTION SYSTEM::STORE> 3 - #< - 0 - #(8 #<SYSTEM-FUNCTION SYSTEM::VECTOR-INIT> #<SYSTEM-FUNCTION SYSTEM::VECTOR-UPD> #<SYSTEM-FUNCTION SYSTEM::VECTOR-ENDTEST> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-INIT> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-UPD> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-ENDTEST> #<SYSTEM-FUNCTION AREF> #<SYSTEM-FUNCTION SYSTEM::STORE> #<SYSTEM-FUNCTION IDENTITY> #<SYSTEM-FUNCTION SYSTEM::VECTOR-LENGTH> #<COMPILED-FUNCTION #:|67 91 (MAPC #'(LAMBDA # #) '(1 2 4 ...))-5-1|> #<SYSTEM-FUNCTION AREF> #<SYSTEM-FUNCTION SYSTEM::STORE> #<SYSTEM-FUNCTION SYSTEM::VECTOR-INIT-START> #<SYSTEM-FUNCTION SYSTEM::VECTOR-FE-INIT-END>) - 3566 - 0 - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D> - #(0 0 0 0 0 ...) <13> #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - (UNSIGNED-BYTE 8) - CHARACTER - #<COMPILED-FUNCTION READ-SEQUENCE> - 3566 - 0 - (:START 0 :END 3566) - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D> - #(0 0 0 0 0 ...) <14> #<COMPILED-FUNCTION READ-SEQUENCE> - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> - 3566 - #<COMPILED-FUNCTION CHUNGA::GET-CHUNK-SIZE> - #(#(#(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>) #<COMPILED-FUNCTION CHUNGA::ADD-EXTENSIONS>) frame binding variables (~ = dynamically): | ~ CHUNGA::*CURRENT-ERROR-FUNCTION* <--> #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1-1|> - #(#(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22CB5D>) - #(NIL #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811>) - #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1|> - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> <15> #<COMPILED-FUNCTION #:|86 143 (DEFMETHOD FILL-BUFFER (#) ...)-9-1-1|> <16> #<STANDARD-GENERIC-FUNCTION CHUNGA::FILL-BUFFER> - #<COMPILED-FUNCTION #:|145 156 (DEFMETHOD STREAM-READ-BYTE (#) ...)-10-1-1|> - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> <17> #<COMPILED-FUNCTION #:|145 156 (DEFMETHOD STREAM-READ-BYTE (#) ...)-10-1-1|> <18> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-BYTE> - NIL - NIL - #<CHUNGA:CHUNKED-IO-STREAM #x1A22A811> <19> #<SYSTEM-FUNCTION READ-BYTE> - #<COMPILED-FUNCTION FLEXI-STREAMS::READ-BYTE*> - NIL - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <20> #<COMPILED-FUNCTION FLEXI-STREAMS::READ-BYTE*> - #<COMPILED-FUNCTION FLEXI-STREAMS::READ-CHAR-8-BIT> - #(0 1 2 3 4 5 ... 253 254 255) - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <21> #<COMPILED-FUNCTION FLEXI-STREAMS::READ-CHAR-8-BIT> - #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE> <22> #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE> - #<COMPILED-FUNCTION FLEXI-STREAMS::GET-CHAR-CODE> - :LF - #(#(NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99> :ISO-8859-1) - #(NIL #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865>) - #<COMPILED-FUNCTION #:|221 275 (DEFMETHOD STREAM-READ-CHAR (#) ...)-11-1-1|> - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <23> #<COMPILED-FUNCTION #:|221 275 (DEFMETHOD STREAM-READ-CHAR (#) ...)-11-1-1|> <24> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-CHAR> - NIL - 0 - #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1-1|> - 8192 - 0 - "^@^@^@^@^@..." - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <25> #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1-1|> - #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1-1|> - #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1|> - NIL - 8192 - 0 - "^@^@^@^@^@..." - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <26> #<COMPILED-FUNCTION #:|285 311 (DEFMETHOD STREAM-READ-SEQUENCE (# SEQUENCE START ...) ...)-13-1-1|> <27> #<STANDARD-GENERIC-FUNCTION TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE> - 8192 - 0 - #<COMPILED-FUNCTION #:|30 105 (PROGN (DEFMETHOD GRAY:STREAM-READ-BYTE-SEQUENCE # ...) (DEFMETHOD GRAY:STREAM-WRITE-BYTE-SEQUENCE # ...) ...)-11-3-1-1|> - (0 8192) - "^@^@^@^@^@..." - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <28> #<COMPILED-FUNCTION #:|30 105 (PROGN (DEFMETHOD GRAY:STREAM-READ-BYTE-SEQUENCE # ...) (DEFMETHOD GRAY:STREAM-WRITE-BYTE-SEQUENCE # ...) ...)-11-3-1-1|> <29> #<STANDARD-GENERIC-FUNCTION GRAY:STREAM-READ-CHAR-SEQUENCE> - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> - "^@^@^@^@^@..." - 8192 - 0 - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> - "^@^@^@^@^@..." <30> #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - #<SYSTEM-FUNCTION EXT:READ-CHAR-SEQUENCE> - CHARACTER - CHARACTER - #<COMPILED-FUNCTION READ-SEQUENCE> - #<UNBOUND> - #<UNBOUND> - NIL - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> - "^@^@^@^@^@ <31> #<COMPILED-FUNCTION READ-SEQUENCE> - NIL - 0 - "" - "^@^@^@^@^@ - T - CHARACTER - NIL - #(NIL ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=iso-8859-1"))) - #<COMPILED-FUNCTION DRAKMA::READ-BODY> - #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99> - T - ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=iso-8859-1")) - #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> <32> #<COMPILED-FUNCTION DRAKMA::READ-BODY> - NIL - #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) #x1A22CB99> - NIL - ((:DATE . "Tue, 24 Oct 2006 02:51:03 GMT") (:SERVER . "Apache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=iso-8859-1")) - 200 - :HTTP/1.1 - #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST> - NIL - NIL <33> #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST> - #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-FINISH-REQUEST> - #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-WRITE-HEADER> - #(#(#(NIL #<URI http://weitz.de/drakma/%3E NIL :HTTP/1.1 :GET NIL NIL NIL 5 (:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> T) #<COMPILED-FUNCTION DRAKMA::HTTP-REQUEST-WRITE-HTTP-LINE>) unwind-protect frame - #(#(NIL #<URI http://weitz.de/drakma/%3E NIL :HTTP/1.1 :GET NIL NIL NIL 5 (:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL) #<FLEXI-STREAMS:FLEXI-IO-STREAM #x1A22C865> T) frame binding variables (~ = dynamically): | ~ FLEXI-STREAMS:*DEFAULT-EOL-STYLE* <--> :LF - NIL - :LF compiled block frame for HTTP-REQUEST - NIL - #(NIL #<URI http://weitz.de/drakma/%3E NIL :HTTP/1.1 :GET NIL NIL NIL 5 (:GET :HEAD) NIL :LATIN-1 :LATIN-1 NIL NIL) - #<COMPILED-FUNCTION HTTP-REQUEST> - NIL - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - T - NIL - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - NIL - NIL - "*/*" - :DRAKMA - NIL - #<UNBOUND> - NIL - NIL - "application/x-www-form-urlencoded" - NIL - NIL - #<UNBOUND> - #<UNBOUND> - #<UNBOUND> - NIL - "http://weitz.de/drakma/" <34> #<COMPILED-FUNCTION HTTP-REQUEST> - #<COMPILED-FUNCTION HTTP-REQUEST> EVAL frame for form (HTTP-REQUEST "http://weitz.de/drakma/") frame binding environments VAR_ENV <--> NIL FUN_ENV <--> NIL BLOCK_ENV <--> NIL GO_ENV <--> NIL DECL_ENV <--> ((DECLARATION OPTIMIZE DECLARATION)) <35> ...

Regards, -Anton