#36: file-position broken for utf16 and utf32 --------------------+------------------------------------------------------- Reporter: rtoy | Owner: somebody Type: defect | Status: new Priority: minor | Milestone: Component: Core | Version: 19f Keywords: | --------------------+------------------------------------------------------- Consider this code:
{{{ (defun bug (&optional (format :utf16)) (with-open-file (s "/tmp/bom.txt" :direction :output :if-exists :supersede :external-format format) (format s "Hello~%")) (with-open-file (s "/tmp/bom.txt" :direction :input :external-format format) (print (read-char s)) (print (file-position s))) (values)) }}}
Running {{{(bug :utf16)}}} produces {{{ #\H 2 }}}
{{{(bug :utf32)}}} produces {{{ #\H 4 }}}
In both cases, the actual position is wrong. For utf16, the position should 4; utf32, 8. The BOM has been ignored.
This is caused by {{{STRING-ENCODE}}} outputting the BOM for these formats. {{{STRING-ENCODE)}}} is used to figure out how many octets have not yet been processed but have been read from the file. If the BOM was not output, the position would be correct.
This bug (will) occur in the 2010-02 snapshot and later.