From lispercat@gmail.com Sun May 20 10:33:20 2007 From: Andrei Stebakov To: drakma-devel@common-lisp.net Subject: [drakma-devel] Help with utf-8 (SBCL 1.0.5) Date: Sun, 20 May 2007 10:33:16 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2669917744909169258==" --===============2669917744909169258== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit When I make the request like this: (drakma:http-request "http://www.w3.org/2001/06/utf-8-test/UTF-8-demo.html") It results in error: " UTF-8 test file

Original by Markus Kuhn, adapted for HTML by Martin Dürst.

UTF-8 encoded sample plain-text file

debugger invoked on a SB-INT:STREAM-ENCODING-ERROR in thread #:
  encoding error on stream #
  (:EXTERNAL-FORMAT :ASCII):
    the character with code 8254 cannot be encoded.

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [OUTPUT-NOTHING] Skip output of this character.
  1: [ABORT         ] Exit debugger, returning to top level.

(SB-INT:STREAM-ENCODING-ERROR
 #
 8254)
0]


Is it some parameter that I forgot to call http-request with to make it
aware of utf-8 encoding?

Thank you,
Andrew



--===============2669917744909169258==
Content-Type: text/html
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="attachment.html"
MIME-Version: 1.0

V2hlbiBJIG1ha2UgdGhlIHJlcXVlc3QgbGlrZSB0aGlzOjxicj4oZHJha21hOmh0dHAtcmVxdWVz
dCAmcXVvdDs8YSBocmVmPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxLzA2L3V0Zi04LXRlc3QvVVRG
LTgtZGVtby5odG1sIj5odHRwOi8vd3d3LnczLm9yZy8yMDAxLzA2L3V0Zi04LXRlc3QvVVRGLTgt
ZGVtby5odG1sPC9hPiZxdW90Oyk8YnI+PGJyPkl0IHJlc3VsdHMgaW4gZXJyb3I6PGJyPgo8YnI+
JnF1b3Q7Jmx0OyFET0NUWVBFIGh0bWwgUFVCTElDIFwmcXVvdDstLy9XM0MvL0RURCBYSFRNTCAx
LjAgVHJhbnNpdGlvbmFsLy9FTlwmcXVvdDs8YnI+Jm5ic3A7Jm5ic3A7Jm5ic3A7IFwmcXVvdDs8
YSBocmVmPSJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRtbDEvRFREL3hodG1sMS10cmFuc2l0aW9u
YWwuZHRkXCI+aHR0cDovL3d3dy53My5vcmcvVFIveGh0bWwxL0RURC94aHRtbDEtdHJhbnNpdGlv
bmFsLmR0ZFwKPC9hPiZxdW90OyZndDs8YnI+Jmx0O2h0bWwgeG1sbnM9XCZxdW90OzxhIGhyZWY9
Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGh0bWxcIj5odHRwOi8vd3d3LnczLm9yZy8xOTk5L3ho
dG1sXDwvYT4mcXVvdDsmZ3Q7PGJyPiZuYnNwOyZuYnNwOyZuYnNwOyAmbHQ7aGVhZCZndDsmbHQ7
dGl0bGUmZ3Q7VVRGLTggdGVzdCBmaWxlJmx0Oy90aXRsZSZndDsmbHQ7L2hlYWQmZ3Q7PGJyPiZu
YnNwOyZuYnNwOyZuYnNwOyAmbHQ7Ym9keSZndDs8YnI+CiZuYnNwOyZuYnNwOyZuYnNwOyAmbHQ7
cCZndDtPcmlnaW5hbCBieSBNYXJrdXMgS3VobiwgYWRhcHRlZCBmb3IgSFRNTCBieSBNYXJ0aW4g
RCZhbXA7dXVtbDtyc3QuJmx0Oy9wJmd0Ozxicj4mbHQ7cHJlJmd0Ozxicj5VVEYtOCBlbmNvZGVk
IHNhbXBsZSBwbGFpbi10ZXh0IGZpbGU8YnI+PGJyPmRlYnVnZ2VyIGludm9rZWQgb24gYSBTQi1J
TlQ6U1RSRUFNLUVOQ09ESU5HLUVSUk9SIGluIHRocmVhZCAjJmx0O1RIUkVBRCAmcXVvdDtpbml0
aWFsIHQKPGJyPmhyZWFkJnF1b3Q7IHtCQkRGNUE5fSZndDs6PGJyPiZuYnNwOyBlbmNvZGluZyBl
cnJvciBvbiBzdHJlYW0gIyZsdDtTQi1TWVM6RkQtU1RSRUFNIGZvciAmcXVvdDtzdGFuZGFyZCBv
dXRwdXQmcXVvdDsge0JCREY4NTl9Jmd0Ozxicj4mbmJzcDsgKDpFWFRFUk5BTC1GT1JNQVQgOkFT
Q0lJKTo8YnI+Jm5ic3A7Jm5ic3A7Jm5ic3A7IHRoZSBjaGFyYWN0ZXIgd2l0aCBjb2RlIDgyNTQg
Y2Fubm90IGJlIGVuY29kZWQuPGJyPjxicj4KVHlwZSBIRUxQIGZvciBkZWJ1Z2dlciBoZWxwLCBv
ciAoU0ItRVhUOlFVSVQpIHRvIGV4aXQgZnJvbSBTQkNMLjxicj48YnI+cmVzdGFydHMgKGludm9r
YWJsZSBieSBudW1iZXIgb3IgYnkgcG9zc2libHktYWJicmV2aWF0ZWQgbmFtZSk6PGJyPiZuYnNw
OyAwOiBbT1VUUFVULU5PVEhJTkddIFNraXAgb3V0cHV0IG9mIHRoaXMgY2hhcmFjdGVyLjxicj4m
bmJzcDsgMTogW0FCT1JUJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7
Jm5ic3A7IF0gRXhpdCBkZWJ1Z2dlciwgcmV0dXJuaW5nIHRvIHRvcCBsZXZlbC4KPGJyPjxicj4o
U0ItSU5UOlNUUkVBTS1FTkNPRElORy1FUlJPUjxicj4mbmJzcDsjJmx0O1NCLVNZUzpGRC1TVFJF
QU0gZm9yICZxdW90O3N0YW5kYXJkIG91dHB1dCZxdW90OyB7QkJERjg1OX0mZ3Q7PGJyPiZuYnNw
OzgyNTQpPGJyPjBdPGJyPjxicj48YnI+SXMgaXQgc29tZSBwYXJhbWV0ZXIgdGhhdCBJIGZvcmdv
dCB0byBjYWxsIGh0dHAtcmVxdWVzdCB3aXRoIHRvIG1ha2UgaXQgYXdhcmUgb2YgdXRmLTggZW5j
b2Rpbmc/Cjxicj48YnI+VGhhbmsgeW91LDxicj5BbmRyZXc8YnI+Cg==

--===============2669917744909169258==--


From rm@seid-online.de Sun May 20 10:42:50 2007
From: Ralf Mattes 
To: drakma-devel@common-lisp.net
Subject: Re: [drakma-devel] Help with utf-8 (SBCL 1.0.5)
Date: Sun, 20 May 2007 16:42:43 +0200
Message-ID: <1179672163.6399.1.camel@localhost.localdomain>
In-Reply-To: 
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============0690566320081686981=="

--===============0690566320081686981==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

sOn Sun, 2007-05-20 at 10:33 -0400, Andrei Stebakov wrote:
> When I make the request like this:
> (drakma:http-request
> "http://www.w3.org/2001/06/utf-8-test/UTF-8-demo.html")
> 
> It results in error:
[...]
> 
> Is it some parameter that I forgot to call http-request with to make
> it aware of utf-8 encoding? 

What does the following eval to:

(if (member :sb-unicode *features*)
    :we-have-unicode :no-unicode-found)


Cheers, Ralf Mattes
 
> Thank you,
> Andrew
> _______________________________________________
> drakma-devel mailing list
> drakma-devel(a)common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel


--===============0690566320081686981==--


From ehuels@gmail.com Sun May 20 11:02:27 2007
From: Erik Huelsmann 
To: drakma-devel@common-lisp.net
Subject: Re: [drakma-devel] Help with utf-8 (SBCL 1.0.5)
Date: Sun, 20 May 2007 17:02:18 +0200
Message-ID: 
In-Reply-To: 
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============1651061356243720987=="

--===============1651061356243720987==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

On 5/20/07, Andrei Stebakov  wrote:
> When I make the request like this:
> (drakma:http-request
> "http://www.w3.org/2001/06/utf-8-test/UTF-8-demo.html")
>
> It results in error:
>
> "
> \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\
> ">
> 
>     UTF-8 test file
>     
>      

Original by Markus Kuhn, adapted for HTML by Martin Dürst.

>
> UTF-8 encoded sample plain-text file
>
> debugger invoked on a SB-INT:STREAM-ENCODING-ERROR in thread # "initial t
> hread" {BBDF5A9}>:
>   encoding error on stream # {BBDF859}>
>   (:EXTERNAL-FORMAT :ASCII):
>     the character with code 8254 cannot be encoded.
>
>  Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
>
> restarts (invokable by number or by possibly-abbreviated name):
>   0: [OUTPUT-NOTHING] Skip output of this character.
>   1: [ABORT         ] Exit debugger, returning to top level.
>
> (SB-INT:STREAM-ENCODING-ERROR
>  #
                                    ^^^^^^^^^^^^^^^

>  8254)
> 0]
>
>
> Is it some parameter that I forgot to call http-request with to make it
> aware of utf-8 encoding?

No. SBCL has detected your terminal doesn't support the characters
you're trying to print, so, it's generating an error.  As you can see
a few lines higher up, the standard output file has an associated
encoding of ASCII, meaning that probably the output of 'locale' on
your system contains LANG=POSIX or LANG=C.

> Thank you,
> Andrew

HTH,

Erik.

--===============1651061356243720987==--