* alexandria-1/io.lisp: (read-stream-content-into-string): fix recent lossage: This addresses problems in commits (13b1575, 6ff6820, 9c97e6f, on 2021-10-24), which changed the element-type of the array used by READ-SEQUENCE to read from the stream from CHARACTER to the STREAM-ELEMENT-TYPE of the stream.
On CCL, ECL etc. this breaks nyxt's source/start.lisp:(listen-socket) which basically does (iolib:with-open-socket (s :address-family :local :connect :passive :local-filename "/tmp/foo") (alexandria:read-stream-content-into-string (iolib:accept-connection s)))
Iolib returns an octet stream for which the STREAM-ELEMENT-TYPE is not CHARACTER. READ-STREAM-CONTENT-INTO-STRING uses WITH-OUTPUT-TO-STRING and that calls MAKE-STRING-OUTPUT-STREAM, the argument :ELEMENT-TYPE ((UNSIGNED-BYTE 8)) which fails because it must be a subtype of character
The present fix is to check if it is not a character stream and force set the element-type to 'character if it is not. --- alexandria-1/io.lisp | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/alexandria-1/io.lisp b/alexandria-1/io.lisp index d3be536..8c93dd6 100644 --- a/alexandria-1/io.lisp +++ b/alexandria-1/io.lisp @@ -65,6 +65,8 @@ (defun read-stream-content-into-string (stream &key (buffer-size 4096)) (check-type buffer-size positive-integer) (let ((*print-pretty* nil) (element-type (stream-element-type stream))) + (unless (subtypep element-type 'character) + (setq element-type 'character)) (with-output-to-string (datum nil :element-type element-type) (let ((buffer (make-array buffer-size :element-type element-type))) (loop
- (unless (subtypep element-type 'character)
(setq element-type 'character))
just a quick note: SUBTYPEP calls can be *very* costly.
This doesn't look like a problem on the Alexandria side. Stuffing ub8 values into any place that requires characters *always* requires additional information about character encoding (is it latin1? latin2? windows-1250?) and dealing with character encodings is out of scope of Alexandria. I think that Flexi Streams/Babel should be explicitly used in this context.
On 20.08.2022 18:35, Madhu wrote:
- alexandria-1/io.lisp: (read-stream-content-into-string): fix recent
lossage: This addresses problems in commits (13b1575, 6ff6820, 9c97e6f, on 2021-10-24), which changed the element-type of the array used by READ-SEQUENCE to read from the stream from CHARACTER to the STREAM-ELEMENT-TYPE of the stream.
On CCL, ECL etc. this breaks nyxt's source/start.lisp:(listen-socket) which basically does (iolib:with-open-socket (s :address-family :local :connect :passive :local-filename "/tmp/foo") (alexandria:read-stream-content-into-string (iolib:accept-connection s)))
Iolib returns an octet stream for which the STREAM-ELEMENT-TYPE is not CHARACTER. READ-STREAM-CONTENT-INTO-STRING uses WITH-OUTPUT-TO-STRING and that calls MAKE-STRING-OUTPUT-STREAM, the argument :ELEMENT-TYPE ((UNSIGNED-BYTE 8)) which fails because it must be a subtype of character
The present fix is to check if it is not a character stream and force set the element-type to 'character if it is not.
alexandria-1/io.lisp | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/alexandria-1/io.lisp b/alexandria-1/io.lisp index d3be536..8c93dd6 100644 --- a/alexandria-1/io.lisp +++ b/alexandria-1/io.lisp @@ -65,6 +65,8 @@ (defun read-stream-content-into-string (stream &key (buffer-size 4096)) (check-type buffer-size positive-integer) (let ((*print-pretty* nil) (element-type (stream-element-type stream)))
- (unless (subtypep element-type 'character)
(setq element-type 'character)) (with-output-to-string (datum nil :element-type element-type) (let ((buffer (make-array buffer-size :element-type element-type))) (loop
* Michał "phoe" Herda phoe@teknik.io 32cf971b-ecc1-d54f-8ea6-4746cddca9ee@teknik.io Wrote on Sun, 21 Aug 2022 11:21:15 +0200
This doesn't look like a problem on the Alexandria side. Stuffing ub8 values into any place that requires characters *always* requires additional information about character encoding (is it latin1? latin2? windows-1250?) and dealing with character encodings is out of scope of Alexandria. I think that Flexi Streams/Babel should be explicitly used in this context.
1. The function contract requires it to return a string which is of type character. 2. Arranging to call WITH-OUTPUT-TO-STRING with an octet backing it is buffer is a bug. 3. Lisps implementation support bivalent streams and alendria (correctly) deferred to the implementations behaviour, and this has been changed.
If we accept your position, READ-STREAM-CONTENT-INTO-STRING function would have to be removed from alexandia, since it cannot support it.
On 22.08.2022 04:05, Madhu wrote:
- The function contract requires it to return a string which is of type character.
The function contract doesn't say anything about accepting a stream whose element type is not a character.
- Lisps implementation support bivalent streams and alendria (correctly) deferred to the implementations behaviour, and this has been changed.
The fact that implementations support bivalent streams as a language extension doesn't mean that trying to stuff octets into strings is portable Common Lisp. The third bullet in Alexandria's README (https://gitlab.common-lisp.net/alexandria/alexandria/-/blob/master/README) states that Alexandria is all about portable CL.
Still, after waking up, I am of two minds about this issue, since there's also the issue of programmer convenience, "don't break userspace", and "if an implementation supports an extension, Alexandria could at least try not to get in the way". Are you able to make an account at https://gitlab.common-lisp.net and post an issue to https://gitlab.common-lisp.net/alexandria/alexandria/-/issues? If not, I'll make it and try to ask for opinion there.
alexandria-devel@common-lisp.net
-
Attila Lendvai -
Madhu -
Michał "phoe" Herda