1 Jun
2010
1 Jun
'10
6:51 a.m.
* Luís Oliveira [2010-05-31 20:57+0200] writes:
On Mon, May 31, 2010 at 1:32 PM, Helmut Eller <heller@common-lisp.net> wrote:
Is this assertion
(assert (equal (cffi:with-foreign-string (data "foobar" :encoding :utf-16) (cffi:foreign-string-to-lisp data :count 6 :encoding :utf-16)) "foobar"))
expected to fail?
FOREIGN-STRING-ALLOC currently doesn't insert BOM markers and the :UTF-16 encoder uses native endianness. The :UTF-16 decoder defaults to big endian when a BOM marker is not present, as recommended by the Unicode specification. That's why the assertion fails.
One solution is to use :UTF-16BE or :UTF-16LE explicitly, which unfortunately aren't implemented in Babel yet.
Thanks. I will write my on decoder then. Helmut