* Raymond Toy [2010-10-02 16:30] writes:
- Introduce variants of length and read-sequence that use the same notion of character as Emacs. Kinda messy and probably slow, but relatively easy.
I don't know slime internals, but wouldn't you only need a special version of length and read-sequence for cmucl with unicode? The normal length/read-sequence would be fine for everyone else.
Well, I wrote the string with prin1 to a file with CMUCL and read it back with
(with-open-file (f "/tmp/x.utf8" :external-format :utf-8) (length (read f)))
in other implementations. The results are CMUCL, Allegro: 4 SBCL, CCL, CLISP: 3, ABCL: 6
Lispworks didn't want to read the file and ECL didn't accept utf-8 (probably pilot error).
ABCL probably ignores the external format argument, but if they use a Java-like representation then it would be 4.
Anyway CMUCL and Allegro have the same problem.
- Switch from character streams to binary streams so that we can use byte counts instead of character counts. This has several advantages:
- surrogate pairs are no problem
- don't need flexi-streams for Lispworks
Why does Lispworks need flexi-streams?
Because Lispworks' socket interface has no (documented) option to specify an encoding. You can write your own stream classes and flexi-streams does that (presumably on top of binary streams).
Does this have to do with using read-byte on character streams or read-char on binary streams?
No. At least SLIME doesn't need that.
- it would be easier to switch encoding after connecting - read/write-sequence is probably faster on byte streamsdisadvantageous: - more consing, and Emacs's GC isn't that good - need a string-to/from-bytearray function for every backend
Doesn't every backend already have such a function? Of course, someone has to hook that up, but at least it doesn't have to be written from scratch.
Yes, most Lisps will have something like that but we still need to figure out how it's called for each one.
- breaks third party backendsSounds like a show stopper to me.
That's the least of my worries.
Helmut