I have been having some trouble using the publicly available flexi-streams package (http://www.cliki.net/flexi-streams). The flexi-streams package is described as follows:
FLEXI-STREAMS is a library which implements "virtual" bivalent streams that can be layered atop real binary/bivalent streams. It can be used to read and write character data in various single- or multi-octet encodings which can be changed on the fly. It also supplies in-memory binary streams which are similar to string streams.
Here's the problem: the flexi-stream library defines a flexi-stream class which ISA stream and has an EXTERNAL-FORMAT slot (and an :external-format initarg):
(defclass flexi-stream (trivial-gray-stream-mixin) ((stream :initarg :stream :reader flexi-stream-stream :documentation "The actual stream that's used for input and/or output. It must be capable of reading/writing octets with READ-SEQUENCE and/or WRITE-SEQUENCE.") (external-format :initform (make-external-format :iso-8859-1) :initarg :external-format :accessor flexi-stream-external-format :documentation "The encoding currently used by this stream. Can be changed on the fly.") (element-type :initform #+:lispworks 'lw:simple-char #-:lispworks 'character :initarg :element-type :accessor flexi-stream-element-type :documentation "The element type of this stream. Must be a subtype of CHARACTER.")) (:documentation "A FLEXI-STREAM object is a stream that's `layered' atop an existing binary/bivalent stream in order to allow for multi-octet external formats. FLEXI-STREAM itself is a mixin and should not be instantiated."))
Flexi-streams allow this initarg to be a list of symbols. Unfortunately, calling MAKE-INSTANCE on a flexi-stream blows up on ACL because ACL has a :BEFORE method for (setf external-format) on STREAM, which rejects the flexi-stream lists that are supplied as initargs. I am not entirely sure why this is being triggered.
Here is an email that specifies what is going wrong, with a simple test case that you can cut out and run under ACL, and a response from the flexi-stream builder.
"RPG" == rpgoldman rpgoldman@real-time.com writes:
"EW" == Edi Weitz edi@agharta.de writes:
RPG> [...snip...]
EW> On Thu, 1 Jun 2006 14:35:26 -0500, "Robert P. Goldman" EW> rpgoldman@sift.info wrote:
>>> I just pulled a copy of flexi-streams, using asdf-install, and >>> had a couple of minor problems: >>>
RPG> [...snip...]
>>> 2. Making a flexible stream with an external format blows up >>> for me on ACL. E.g., cl-irc tries to make a flexi-io-stream >>> with external format (:utf-8 :eof-type :crlf) >>> >>> This value is stuffed into the flexi-stream slot >>> EXTERNAL-FORMAT. Unfortunately, IIUC, this also triggers an >>> Allegro-specific method as follows: >>> >>> (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE (T STREAM)) >>> >>> This method checks to make sure that the value you are setting >>> as a stream-external-format is an excl:external-format object. >>> In this case, it clearly isn't, so the :before method hurls an >>> error. >>> >>> I believe that this error is triggered because a >>> flexi-io-stream ISA fundamental-binary-input-stream which ISA >>> stream. >>> >>> I confess I'm not really sure what to do about this >>> problem....
EW> Hmm, at first glance I'd say that this is either an AllegroCL EW> bug or it is not related to FLEXI-STREAMS at all. The slot EW> you're talking about above is called EW> FLEXI-STREAMS::EXTERNAL-FORMAT, its name is not exported from EW> the package. I can't see why setting the value of this slot EW> should trigger an Allegro-specific method.
EW> (Note that on AllegroCL EXCL:EXTERNAL-FORMAT is accessible in EW> various packages, including CL-USER. Maybe you're seeing a EW> package conflict?)
RPG> The problem is not the FLEXI-STREAMS::EXTERNAL-FORMAT symbol, RPG> but the use of :external-format as an initarg, IIUC. See the RPG> (relatively) simple test case below for more details.
EW> It'd be nice if you could provide a simple test case which EW> provokes this error message, so we can see what's really going EW> on.
RPG> OK. Understand that I don't really know how flexi-streams RPG> are supposed to work --- I just stumbled into this because I RPG> use BEIRC, which uses CL-IRC, which (newly) uses RPG> FLEXI-STREAMS. So I'm pretty far from understanding things, RPG> and some of the things that I give may not be bugs; they may RPG> be me using the package wrong.
RPG> My first shot at causing an error is the following:
RPG> (defpackage :flexi-test (:use :common-lisp :flex)) RPG> (in-package :flexi-test) (setf foo (open "/tmp/foo" RPG> :direction :io)) (setf bar (make-flexi-stream foo)) Error: RPG> :DEFAULT is not known to be a name for an external format.
RPG> Restart actions (select using :continue): RPG> 0: Return to Top Level (an "abort" restart). 1: Abort RPG> entirely from this (lisp) process. RPG> [1] FLEXI-TEST(18): :bt Evaluation stack:
RPG> FLEX::NORMALIZE-EXTERNAL-FORMAT-NAME <- RPG> FLEX::MAKE-EXTERNAL-FORMAT% <- MAKE-EXTERNAL-FORMAT <- RPG> FLEX::MAYBE-CONVERT-EXTERNAL-FORMAT <- (METHOD RPG> INITIALIZE-INSTANCE :AFTER ...) <- (:INTERNAL RPG> (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE RPG> (CLASS)) <- (METHOD MAKE-INSTANCE (SYMBOL)) <- RPG> MAKE-FLEXI-STREAM <- [... EXCL::%EVAL ] <- LET* <- RPG> [... EXCL::%EVAL ] <- EVAL <- RPG> TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP <- RPG> TPL:START-INTERACTIVE-TOP-LEVEL
RPG> [I'm honestly not sure where the :default value comes from, RPG> and it may be that I am simply committing an error here and RPG> that the :external-format keyword is mandatory, rather than RPG> optional, as I have treated it.]
RPG> So I tried to do what cl-irc does:
RPG> (defun external-format-fixup (format) RPG> (let ((new-format (copy-list format))) RPG> (setf (getf (cdr new-format) :eol-style) :crlf) RPG> new-format)) RPG> (defun mock-connect (&key (server "irc.freenode.net") RPG> (port 6667) RPG> (connection-type RPG> 'connection) RPG> (logging-stream t)) RPG> (let* ((stream (socket-connect server port)) RPG> (connection (make-connection RPG> :connection-type connection-type RPG> :network-stream stream RPG> :client-stream RPG> logging-stream RPG> :server-name server))) RPG> (values connection stream))) RPG> (defun make-connection (&key (connection-type 'connection) RPG> (user nil) (password nil) RPG> (server-name "") (server-port RPG> nil) (network-stream nil) RPG> (outgoing-external-format RPG> *default-outgoing-external-format*) RPG> (client-stream t) (hooks nil)) RPG> (let ((output-stream (flexi-streams:make-flexi-stream RPG> network-stream :element-type RPG> 'character :external-format RPG> (external-format-fixup RPG> outgoing-external-format)))) RPG> output-stream)) RPG> ;; overlooked this dependency.... RPG> (asdf:oos 'asdf:load-op :trivial-sockets) (defun RPG> socket-connect (server port) RPG> "Create a socket connected to `server':`port' and return RPG> stream for it." (trivial-sockets:open-stream server port RPG> :element-type '(unsigned-byte 8))) RPG> (setf *default-outgoing-external-format* '(:utf8)) That gets RPG> me the error: FLEXI-TEST(62): (mock-connect) Error: RPG> `:EOL-STYLE' does not name an external-format. RPG> [condition type: NO-EXTERNAL-FORMAT-ERROR]
RPG> Restart actions (select using :continue): RPG> 0: Return to Top Level (an "abort" restart). 1: Abort RPG> entirely from this (lisp) process. RPG> [1] FLEXI-TEST(63): :bt Evaluation stack:
RPG> (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE ...) <- RPG> (:INTERNAL (:EFFECTIVE-METHOD 2 NIL ...) 0) <- (:INTERNAL RPG> (:EFFECTIVE-METHOD 2 T ...) 0) <- (METHOD RPG> INITIALIZE-INSTANCE (STANDARD-OBJECT)) <- (:INTERNAL RPG> (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE RPG> (CLASS)) <- (METHOD MAKE-INSTANCE (SYMBOL)) <- RPG> MAKE-FLEXI-STREAM <- LET <- MAKE-CONNECTION <- LET* <- RPG> MOCK-CONNECT <- EVAL <- TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP RPG> <- TPL:START-INTERACTIVE-TOP-LEVEL
RPG> The error happens when making the flexi-io-stream class. It RPG> triggers this ACL :before method that checks the value of the RPG> stream-external-format, and which seems to be seeing the RPG> '(:utf8 :eol-style :crlf) value that cl-irc is stuffing into RPG> the flexi-stream constructor.
RPG> ACL has a method that evidently checks the assignment to RPG> stream-external-format for every stream to make sure that it RPG> is a legitimate excl-external-format, IIUC. A flexi-stream RPG> ISA stream, so somehow this is triggered. Your email seems RPG> to suggest that you did NOT expect the flexi-stream-value to RPG> become the stream-external-format, right? But this is the RPG> make-instance call:
RPG> ((METHOD MAKE-INSTANCE (CLASS)) #<STANDARD-CLASS RPG> FLEXI-IO-STREAM> :STREAM RPG> #<MULTIVALENT stream socket connected from RPG> #192.168.1.18/41332 to RPG> kornbluth.freenode.net/6667 @ #x71dc7412> RPG> :ELEMENT-TYPE CHARACTER :EXTERNAL-FORMAT (:UTF8 :EOL-STYLE RPG> :CRLF))
RPG> So even though flexi-streams::external-format is not RPG> exported, flexi-streams is using the :external-format keyword RPG> in its make-instance call, and that is triggering the ACL RPG> code. Would it be enough to fix things to change the RPG> :initarg to ":flexi-stream-external-format"? That seems RPG> cumbersome, though. I don't know another way to keep ACL RPG> from grabbing up the :external-format keyword arg, though... RPG> But again, I have not thought deeply about this.
"EW" == Edi Weitz edi@agharta.de writes:
EW> I think you're basically right and what happens is this:
CL-USER> (defclass foo (excl::fundamental-character-stream) EW> ((bar :initarg :external-format EW> :initform (error "No external EW> format.")))) EW> #<STANDARD-CLASS FOO> CL-USER> (make-instance 'foo) EW> #<FOO @ #x2122c9aa> CL-USER> (slot-value * 'bar) EW> :DEFAULT
EW> So, it looks like there's some :AROUND method within AllegroCL EW> which modifies the initargs before the FOO instance is EW> initialized. I couldn't find a place in their documentation EW> where they explicitely talk about the :EXTERNAL-FORMAT EW> initarg, and I'm hesitant to rename it in FLEXI-STREAMS EW> because it is the right name, isn't it?
EW> At least, before I do that, it'd be nice if you could ask EW> Franz about their take.
EW> Cheers, Edi.