On Mon, 9 Jun 2008 22:20:50 -0600, "Wade Humeniuk" wade.humeniuk@gmail.com wrote:
I am having trouble with Clozure CL. I am not sure this is the right mailing list as it seems to be a CCL problem (unexpected end-of-file). This same request works with LW 5.0.1 and SBCL 1.0.15 on the same machine. CCL also works with other urls like " https://bmo.com". The failing url is being served by an apache web server. I have noticed similiar problems have been posted to this mailing list.
I can confirm that this fails for me with ClozureCL and works with LispWorks. A short analysis of what I think happens:
1. We're reading from a text stream with UTF-8 encoding.
2. The server tells us the length of the body is 2353 octets.
3. Drakma creates a string of length 2353. With UTF-8, the string could be shorter, but you can't be sure before you've read it.
4. It then calls READ-SEQUENCE on this string which is implemented by FLEXI-STREAMS.
5. FLEXI-STREAMS, knowing that this is UTF-8 and /not/ knowing that there can be at most 2353 octets (2353 /characters/ are asked for) tries to read a bit more. It tries to read 2355 octets from the underlying binary stream.
6. The underlying stream is a Chunga stream, but as there's no chunked encoding involved, READ-SEQUENCE is directly invoked on the SSL stream.
7. That's where the error happens.
My understanding is that if you're trying to READ-SEQUENCE 2355 octets and there's an EOF after 2353 octets, then READ-SEQUENCE should read these 2353 octets and return the position as described by the ANSI standard. I don't think it is correct that ClozureCL signals an error here. You should probably ask on their mailing list. (Does this also happen with HTTP instead of HTTPS? Maybe it's a problem with the CL+SSL library?)
A workaround would be to send the request with :FORCE-BINARY T and to convert the content yourself which is admittedly a kludge.
Having said that, this is actually due to the way READ-SEQUENCE is implemented in newer versions of FLEXI-STREAMS. Now that FLEXI-STREAMS has an efficient version of OCTETS-TO-STRING, I'm wondering if it would make sense if Drakma always read the sequence as binary data and then called OCTETS-TO-STRING on the result if needed. Comments?
Cheers, Edi.