I am having trouble with Clozure CL. I am not sure this is the right mailing list as it seems to be a CCL problem (unexpected end-of-file). This same request works with LW 5.0.1 and SBCL 1.0.15 on the same machine. CCL also works with other urls like " https://bmo.com". The failing url is being served by an apache web server. I have noticed similiar problems have been posted to this mailing list. (setting chunga:*accept-bogus-eols* does not help).
CL-USER> (lisp-implementation-version) "Version 1.2-r9226-RC1 (LinuxX8664)" CL-USER> CL-USER> (drakma:http-request "https://esqa.moneris.com")
Unexpected end of file on #<SSL-STREAM for 4> [Condition of type END-OF-FILE]
Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT-BREAK] Reset this process 2: [ABORT] Kill this process
Backtrace: 0: (READ-BYTE #<SSL-STREAM for 4> 'T 'NIL) 1: (#<CCL::STANDARD-KERNEL-METHOD STREAM-READ-VECTOR (CCL::BINARY-INPUT-STREAM T T T)> #<SSL-STREAM for 4> #(60 72 84 77 76 62 10 60 72 69 ...) 0 2355) 2: (READ-SEQUENCE #(60 72 84 77 76 62 10 60 72 69 ...) #<SSL-STREAM for 4> ':START 0 ':END 2355) 3: (READ-SEQUENCE #(60 72 84 77 76 62 10 60 72 69 ...) #<CHUNGA:CHUNKED-IO-STREAM #x30004141A8FD> ':START 0 ':END 2355) 4: ((:INTERNAL FLEXI-STREAMS::FILL-BUFFER (TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE (FLEXI-STREAMS:FLEXI-INPUT-STREAM T T T))) 2355) 5: (#<STANDARD-METHOD TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE (FLEXI-STREAMS:FLEXI-INPUT-STREAM T T T)> #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004129C14D> 6: (CCL::%%STANDARD-COMBINED-METHOD-DCODE '((#<STANDARD-METHOD TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE :BEFORE (FLEXI-STREAMS:FLEXI-IO-STREAM
T
T
T)>) NIL #<STANDARD-METHOD TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE (FLEXI-STREAMS:FLEXI-INPUT-STREAM T T T)>) 17575205018776) 7: (READ-SEQUENCE <edited> 8: (DRAKMA::READ-BODY #<FLEXI-STREAMS:FLEXI-IO-STREAM #x30004129C14D> '((:DATE . "Tue, 10 Jun 2008 04:15:25 GMT") (:SERVER . "Apache") (:LAST-MODIFIED . "Thu, 08 Nov 2007 15:41:34 GMT") (:ETAG . ""42951-931-b3656780"") (:ACCEPT-RANGES . "bytes") (:CONTENT-LENGTH . "2353") (:CONNECTION . "close") (:CONTENT-TYPE . "text/html; charset=UTF-8")) 'T #<FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) #x30004129BCCD>) 9: ((:INTERNAL DRAKMA::FINISH-REQUEST DRAKMA:HTTP-REQUEST) 'NIL 'NIL) 10: (DRAKMA:HTTP-REQUEST #<URI https://esqa.moneris.com%3E) 11: (CCL::CALL-CHECK-REGS 'DRAKMA:HTTP-REQUEST "https://esqa.moneris.com") 12: (CCL::RUN-PROCESS-INITIAL-FORM #<PROCESS worker(7) [Active] #x30004130E5FD> '(#<COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x30004130E48F>)) 13: ((:INTERNAL CCL::%PROCESS-PRESET-INTERNAL) #<PROCESS worker(7) [Active] #x30004130E5FD> '(#<COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x30004130E48F>)) 14: ((:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION))
On Mon, 9 Jun 2008 22:20:50 -0600, "Wade Humeniuk" wade.humeniuk@gmail.com wrote:
I am having trouble with Clozure CL. I am not sure this is the right mailing list as it seems to be a CCL problem (unexpected end-of-file). This same request works with LW 5.0.1 and SBCL 1.0.15 on the same machine. CCL also works with other urls like " https://bmo.com". The failing url is being served by an apache web server. I have noticed similiar problems have been posted to this mailing list.
I can confirm that this fails for me with ClozureCL and works with LispWorks. A short analysis of what I think happens:
1. We're reading from a text stream with UTF-8 encoding.
2. The server tells us the length of the body is 2353 octets.
3. Drakma creates a string of length 2353. With UTF-8, the string could be shorter, but you can't be sure before you've read it.
4. It then calls READ-SEQUENCE on this string which is implemented by FLEXI-STREAMS.
5. FLEXI-STREAMS, knowing that this is UTF-8 and /not/ knowing that there can be at most 2353 octets (2353 /characters/ are asked for) tries to read a bit more. It tries to read 2355 octets from the underlying binary stream.
6. The underlying stream is a Chunga stream, but as there's no chunked encoding involved, READ-SEQUENCE is directly invoked on the SSL stream.
7. That's where the error happens.
My understanding is that if you're trying to READ-SEQUENCE 2355 octets and there's an EOF after 2353 octets, then READ-SEQUENCE should read these 2353 octets and return the position as described by the ANSI standard. I don't think it is correct that ClozureCL signals an error here. You should probably ask on their mailing list. (Does this also happen with HTTP instead of HTTPS? Maybe it's a problem with the CL+SSL library?)
A workaround would be to send the request with :FORCE-BINARY T and to convert the content yourself which is admittedly a kludge.
Having said that, this is actually due to the way READ-SEQUENCE is implemented in newer versions of FLEXI-STREAMS. Now that FLEXI-STREAMS has an efficient version of OCTETS-TO-STRING, I'm wondering if it would make sense if Drakma always read the sequence as binary data and then called OCTETS-TO-STRING on the result if needed. Comments?
Cheers, Edi.
After reading the Hyperspec I do think it is problem with CCL's read-sequence. I modifed CCL's source from (the code looks obviously in error)
(defmethod stream-read-vector ((stream binary-input-stream) vector start end) (declare (fixnum start end)) (do* ((i start (1+ i))) ((= i end) end) (declare (fixnum i)) (let* ((b (read-byte stream)) (if (eq b :eof) (return i) (setf (uvref vector i) b)))))
to
(defmethod stream-read-vector ((stream binary-input-stream) vector start end) (declare (fixnum start end)) (do* ((i start (1+ i))) ((= i end) end) (declare (fixnum i)) (let* ((b (read-byte stream nil :eof))) (if (eq b :eof) (return i) (setf (uvref vector i) b)))))
and it now works for "https://esqa.moneris.com"
Just for your info Drakma is being used for Moneris credit card purchase validatons.
Example:
CL-USER> (moneris-test::test-purchase) (:RECEIPT (:RECEIPTID "ORDER-3422131578-72") (:REFERENCENUM "660021630017911950") (:RESPONSECODE "027") (:ISO "01") (:AUTHCODE "337410") (:TRANSTIME "20:06:19") (:TRANSDATE "2008-06-10") (:TRANSTYPE "00") (:COMPLETE T) (:MESSAGE "APPROVED * =") (:TRANSAMOUNT "10.00") (:CARDTYPE "V") (:TRANSID "420704-0_7") (:TIMEDOUT NIL) (:BANKTOTALS NIL) (:TICKET NIL)) 27 "APPROVED * =" "420704-0_7" "<?xml version=\"1.0\" standalone=\"yes\"?><response><receipt><ReceiptId>ORDER-3422131578-72</ReceiptId><ReferenceNum>660021630017911950</ReferenceNum><ResponseCode>027</ResponseCode><ISO>01</ISO><AuthCode>337410</AuthCode><TransTime>20:06:19</TransTime><TransDate>2008-06-10</TransDate><TransType>00</TransType><Complete>true</Complete><Message>APPROVED * =</Message><TransAmount>10.00</TransAmount><CardType>V</CardType><TransID>420704-0_7</TransID><TimedOut>false</TimedOut><BankTotals>null</BankTotals><Ticket>null</Ticket></receipt></response>" CL-USER>
I will submit the info to the ClozureCL devel list.
Thanks, Wade
Gary Byers at Clozure has made the fix (as well as a couple of others). You can update with svn update to at least r9730.
Wade
Clozure Common Lisp Version 1.2-r9736M-RC1 (LinuxX8664)
-
Edi Weitz -
Wade Humeniuk