July 2008 - flexi-streams-devel@common-lisp.net

[flexi-streams-devel] New release 1.0.4
by Edi Weitz 25 Jul '08

25 Jul '08

ChangeLog: Version 1.0.4 2008-07-25 Cosmetic surgery on test suite Download: http://weitz.de/files/flexi-streams.tar.gz

1 0

[flexi-streams-devel] flexi-streams 1.0.3 and CMUCL 19e
by Eugene Ossintsev 16 Jul '08

16 Jul '08

Hi, I was trying to compile flexi-streams 1.0.3 with CMUCL 19e and it stopped with the following message in ascii.lisp : -------------------------------------------------------- Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER: 65533 is not of type (UNSIGNED-BYTE 8) [Condition of type TYPE-ERROR] -------------------------------------------------------- As I see, in MAKE-DECODING-TABLE the function MAKE-ARRAY has :element-type 'char-code-integer. And CHAR-CODE-INTEGER is defined as -------------------------------------------------------- (deftype char-code-integer () "The subtype of integers which can be returned by the function CHAR-CODE." '(integer 0 #.(1- char-code-limit))) -------------------------------------------------------- But in CMUCL CHAR-CODE-LIMIT is 256, that is not enough to hold 65533. If I modify mapping.lisp the following way (see also the attached patch): -------------------------------------------------------- (deftype char-code-integer () "The subtype of integers which can be returned by the function CHAR-CODE." - '(integer 0 #.(1- char-code-limit))) + #-:cmu '(integer 0 #.(1- char-code-limit)) + #+:cmu '(integer 0 65533)) -------------------------------------------------------- it's compiled with CMUCL successfully and a simple Hunchentoot web application of mine runs without any problem. But if I try to perform the tests from flexi-streams/test I get a bunch of successful results -------------------------------------------------------- FLEXI-STREAMS-TEST> (run-tests) Test (using COPY-STREAM) "kafka_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT] --> (:UTF-8 :EOL-STYLE :LF) [OUTPUT]. Test (using COPY-STREAM) "kafka_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [IO] --> (:UTF-8 :EOL-STYLE :LF) [OUTPUT]. Test (using COPY-STREAM) "kafka_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT] --> (:UTF-8 :EOL-STYLE :LF) [IO]. [[ skipped; too many to put them all here ]] -------------------------------------------------------- followed by some failures (all with the hebrew* files): -------------------------------------------------------- Test (using COPY-STREAM) "hebrew_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT] --> (:UTF-8 :EOL-STYLE :LF) [OUTPUT]. Test failed!!! [[ skipped ]] Test (using COPY-STREAM) "hebrew_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [IO] --> (:UTF-8 :EOL-STYLE :CRLF) [IO]. Test failed!!! Test (using COPY-STREAM) "hebrew_utf8_lf.txt" (:UTF-8 :EOL-STYLE :LF) [INPUT] --> (:ISO-8859-8 :EOL-STYLE :LF) [OUTPUT]. -------------------------------------------------------- and eventually, hello the debugger! -------------------------------------------------------- #\å (code NIL) is not in this encoding. [Condition of type EXTERNAL-FORMAT-ENCODING-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT] Return to Top-Level. Backtrace: 0: (FLEXI-STREAMS::SIGNAL-ENCODING-ERROR #<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}> "~S (code ~A) is not in this encoding." #\å NIL) 1: (FLEXI-STREAMS::WRITE-SEQUENCE* #<#1=unavailable-arg> #<#1#> #<#1#> #<#1#> ...) 2: (TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE #<#1=unavailable-arg> #<#1#> #<#1#> #<#1#> ...) 3: (LISP::WRITE-LINE* ":åèÐÔ êÐÕ ÝÙÞéÔ êÐ ÝÙÔÜÐ ÐèÑ êÙéÐèÑ Ð 1" #<FLEXI-OUTPUT-STREAM {59392C6D}> 0 41) 4: (COPY-STREAM #<Stream for file "/home/eugene/lisp/flexi-streams-1.0.3/test/hebrew_utf8_lf.txt"> #<FLEXI-STREAMS::FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) {5A537D75}> #<Stream for file "/tmp/odd-streams-test/hebrew_latin8_lf.txt"> #<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}>) 5: (COPY-FILE #P"/home/eugene/lisp/flexi-streams-1.0.3/test/hebrew_utf8_lf.txt" #<FLEXI-STREAMS::FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) {5A537D75}> #P"/tmp/odd-streams-test/hebrew_latin8_lf.txt" #<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}> ...) 6: (COMPARE-FILES "hebrew_utf8_lf.txt" #<FLEXI-STREAMS::FLEXI-UTF-8-FORMAT (:UTF-8 :EOL-STYLE :LF) {5A537D75}> "hebrew_latin8_lf.txt" #<FLEXI-STREAMS::FLEXI-8-BIT-FORMAT (:ISO-8859-8 :EOL-STYLE :LF) {5A537DA5}>) 7: (RUN-TESTS) --more-- -------------------------------------------------------- I don't know if it's the same or similar for the other Common Lisp implementations, but that patch allows flexi-streams at least to be compiled with CMUCL. I know it's far from being perfect. I hope there are much more experienced people here, who know how to fix it. - Eugene

1 0

[flexi-streams-devel] Re: flexi-streams 1.0.3 and CMUCL 19e
by Edi Weitz 14 Jul '08

14 Jul '08

Hi Eugene, On Sun, 13 Jul 2008 22:23:20 -0400, Eugene Ossintsev <eugoss(a)gmail.com> wrote: > I was trying to compile flexi-streams 1.0.3 with CMUCL 19e and it > stopped with the following message in ascii.lisp : > > -------------------------------------------------------- > Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER: > 65533 is not of type (UNSIGNED-BYTE 8) > [Condition of type TYPE-ERROR] > -------------------------------------------------------- > > I don't know if it's my fault or not, I'm not experienced enough > yet. But as I see, in MAKE-DECODING-TABLE the function MAKE-ARRAY > has :element-type 'char-code-integer. > > CHAR-CODE-INTEGER is defined as > > (deftype char-code-integer () > "The subtype of integers which can be returned by the function CHAR-CODE." > '(integer 0 #.(1- char-code-limit))) > > But at least in CMUCL 19e CHAR-CODE-LIMIT is 256, that is not enough > to hold 65533. I'm afraid it's about Unicode-non-Unicode characters, > right? Please send but reports to the mailing list (see Cc). I'm not sure what the best thing to do is w.r.t. CMUCL. Should we declare CHAR-CODE-INTEGER to be the same as FIXNUM on CMUCL? Or (integer 0 65535)? Or should at least MAKE-DECODING-TABLE be modified? Opinions? Cheers, Edi. PS: Eugene, as a workaround please try the following in mapping.lisp and report to the list if you still have problems: (defun make-decoding-table (list) "Creates and returns an array which contains the elements in the list LIST and has an element type that's suitable for character codes." (make-array (length list) :element-type 'fixnum :initial-contents list))

1 0