- flexi-streams-devel@lists.common-lisp.net

[flexi-streams-devel] New version 0.6.1
by Edi Weitz 15 Sep '06

15 Sep '06

ChangeLog: Version 0.6.1 2006-09-15 Switched FILE-POSITION implementation to TRIVIAL-GRAY-STREAMS (thanks to David Lichteblau) Download: http://weitz.de/files/flexi-streams.tar.gz Note: At the moment you'll need the CVS version of TRIVIAL-GRAY-STREAMS. Cheers, Edi.

1 0

[flexi-streams-devel] New version 0.6.0 / FILE-POSITION
by Edi Weitz 13 Sep '06

13 Sep '06

This new version tries to implement a somewhat reasonable behaviour for FILE-POSITION but due to this not being mentioned in the Gray streams proposal, I currently only have code for LispWorks. I'll gladly accept patches to make this work for other implementations as well. http://weitz.de/flexi-streams/#position Cheers, Edi.

1 0

[flexi-streams-devel] New version 0.5.10
by Edi Weitz 04 Sep '06

04 Sep '06

ChangeLog: Version 0.5.10 2006-09-04 Flexi streams can have binary element types now Download: http://weitz.de/files/flexi-streams.tar.gz Cheers, Edi.

1 0

[flexi-streams-devel] New version 0.5.9
by Edi Weitz 03 Sep '06

03 Sep '06

ChangeLog: Version 0.5.9 2006-09-01 Added string functions Download: http://weitz.de/files/flexi-streams.tar.gz Cheers, Edi.

1 0

[flexi-streams-devel] New version 0.5.8
by Edi Weitz 01 Sep '06

01 Sep '06

ChangeLog: Version 0.5.8 2006-09-01 CHECK-EOF-NO-HANG is not necessary Updated LW links in documentation Changed package handling in system definition (thanks to Christophe Rhodes) Download: http://weitz.de/files/flexi-streams.tar.gz Cheers, Edi.

1 0

[flexi-streams-devel] Re: [Sbcl-devel] SB-KERNEL:LAYOUT-INVALID errors with flexi-stream in-memory streams
by James Bielman 04 Aug '06

04 Aug '06

Christophe Rhodes <csr21(a)cam.ac.uk> writes: > I don't have flexi-streams to hand; is there any oddness in > make-in-memory-input-stream? Or does it simply hand-off to > make-instance? It hands off to MAKE-INSTANCE: (defmethod make-in-memory-input-stream ((vector vector) &optional (start 0) (end (length vector))) "Returns a binary input stream which will supply, in order, the octets in the subsequence of VECTOR bounded by START and END." (make-instance 'vector-input-stream :vector vector :index start :end end)) and the class (FWIW): * (describe (find-class 'flexi-streams::vector-input-stream)) #<STANDARD-CLASS FLEXI-STREAMS::VECTOR-INPUT-STREAM> is a class. It is an instance of STANDARD-CLASS. Its proper name is FLEXI-STREAMS::VECTOR-INPUT-STREAM. The direct superclasses are: (FLEXI-STREAMS:VECTOR-STREAM FLEXI-STREAMS:IN-MEMORY-INPUT-STREAM), and the direct subclasses are: (). The class is finalized; its class precedence list is: (FLEXI-STREAMS::VECTOR-INPUT-STREAM FLEXI-STREAMS:VECTOR-STREAM FLEXI-STREAMS:IN-MEMORY-INPUT-STREAM FLEXI-STREAMS:IN-MEMORY-STREAM TRIVIAL-GRAY-STREAMS:TRIVIAL-GRAY-STREAM-MIXIN FUNDAMENTAL-BINARY-INPUT-STREAM FUNDAMENTAL-INPUT-STREAM FUNDAMENTAL-BINARY-STREAM FUNDAMENTAL-STREAM STANDARD-OBJECT SB-PCL::SLOT-OBJECT STREAM T). There are 8 methods specialized for this class. I'm still trying to pull out a self-contained testcase but having no luck. Thanks, James

1 0

[flexi-streams-devel] New version 0.5.7
by Edi Weitz 29 Jun '06

29 Jun '06

ChangeLog: Version 0.5.7 2006-06-29 Removed incompatibility with AllegroCL, see mailing list archive for details Download: http://weitz.de/files/flexi-streams.tar.gz

1 0

[flexi-streams-devel] Re: [spr31726] Issue with flexi-streams
by Edi Weitz 14 Jun '06

14 Jun '06

On Wed, 14 Jun 2006 11:02:13 -0700, "Charles A. Cox" <cox(a)franz.com> wrote: > Thanks for your report. > > This is the first time I've heard of "flexi-streams". From the > description in your message, my first thought is that Allegro CL > users are better off using Allegro CL streams since Allegro CL > streams provide what "flexi-streams" purports to offer (bivalency, > dynamically switchable external-formats, etc.). > > Still, if I understand correctly, there is a specific question of > how the :external-format slot is being initialized in subclasses of > excl:fundamental-character-stream. That class has a > default-initargs which specifies :external-format. Here is a > simplified example to demonstrate what's happening: > > (defclass my-stream () > ((external-format :initarg :external-format))) > > (defclass my-fundamental-character-stream (my-stream) > () > (:default-initargs :external-format :default)) > > [your test case:] > (defclass foo (my-fundamental-character-stream) > ((bar :initarg :external-format > :initform (error "No external format.")))) > > (slot-value (make-instance 'foo) 'bar) ==> :default > > Even though we may not be documenting the external-format slot in > Allegro CL streams, we do document stream-external-format, which is > an accessor to that slot. > > My recommendation is that if flexi-streams implements an > external-format separate from that of the base Lisp, that it then > use a name other than "external-format". Well, you've heard what they say - AllegroCL's streams are better anyway... :) My take still is that if AllegroCL provides classes that can be subclassed and instantiated by user code, then they should either use internal symbols for their initargs or, if they use keyword symbols, they should document them. Anyway, Franz' reaction doesn't really make me feel as if I should hurry to make FLEXI-STREAMS work with AllegroCL. Cheers, Edi.

1 0

[flexi-streams-devel] Issue with flexi-streams
by rpgoldman＠real-time.com 13 Jun '06

13 Jun '06

I have been having some trouble using the publicly available flexi-streams package (http://www.cliki.net/flexi-streams) The flexi-streams package is described as follows: FLEXI-STREAMS is a library which implements "virtual" bivalent streams that can be layered atop real binary/bivalent streams. It can be used to read and write character data in various single- or multi-octet encodings which can be changed on the fly. It also supplies in-memory binary streams which are similar to string streams. Here's the problem: the flexi-stream library defines a flexi-stream class which ISA stream and has an EXTERNAL-FORMAT slot (and an :external-format initarg): (defclass flexi-stream (trivial-gray-stream-mixin) ((stream :initarg :stream :reader flexi-stream-stream :documentation "The actual stream that's used for input and/or output. It must be capable of reading/writing octets with READ-SEQUENCE and/or WRITE-SEQUENCE.") (external-format :initform (make-external-format :iso-8859-1) :initarg :external-format :accessor flexi-stream-external-format :documentation "The encoding currently used by this stream. Can be changed on the fly.") (element-type :initform #+:lispworks 'lw:simple-char #-:lispworks 'character :initarg :element-type :accessor flexi-stream-element-type :documentation "The element type of this stream. Must be a subtype of CHARACTER.")) (:documentation "A FLEXI-STREAM object is a stream that's `layered' atop an existing binary/bivalent stream in order to allow for multi-octet external formats. FLEXI-STREAM itself is a mixin and should not be instantiated.")) Flexi-streams allow this initarg to be a list of symbols. Unfortunately, calling MAKE-INSTANCE on a flexi-stream blows up on ACL because ACL has a :BEFORE method for (setf external-format) on STREAM, which rejects the flexi-stream lists that are supplied as initargs. I am not entirely sure why this is being triggered. Here is an email that specifies what is going wrong, with a simple test case that you can cut out and run under ACL, and a response from the flexi-stream builder. >>>>> "RPG" == rpgoldman <rpgoldman(a)real-time.com> writes: >>>>> "EW" == Edi Weitz <edi(a)agharta.de> writes: RPG> [...snip...] EW> On Thu, 1 Jun 2006 14:35:26 -0500, "Robert P. Goldman" EW> <rpgoldman(a)sift.info> wrote: >>> I just pulled a copy of flexi-streams, using asdf-install, and >>> had a couple of minor problems: >>> RPG> [...snip...] >>> 2. Making a flexible stream with an external format blows up >>> for me on ACL. E.g., cl-irc tries to make a flexi-io-stream >>> with external format (:utf-8 :eof-type :crlf) >>> >>> This value is stuffed into the flexi-stream slot >>> EXTERNAL-FORMAT. Unfortunately, IIUC, this also triggers an >>> Allegro-specific method as follows: >>> >>> (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE (T STREAM)) >>> >>> This method checks to make sure that the value you are setting >>> as a stream-external-format is an excl:external-format object. >>> In this case, it clearly isn't, so the :before method hurls an >>> error. >>> >>> I believe that this error is triggered because a >>> flexi-io-stream ISA fundamental-binary-input-stream which ISA >>> stream. >>> >>> I confess I'm not really sure what to do about this >>> problem.... EW> Hmm, at first glance I'd say that this is either an AllegroCL EW> bug or it is not related to FLEXI-STREAMS at all. The slot EW> you're talking about above is called EW> FLEXI-STREAMS::EXTERNAL-FORMAT, its name is not exported from EW> the package. I can't see why setting the value of this slot EW> should trigger an Allegro-specific method. EW> (Note that on AllegroCL EXCL:EXTERNAL-FORMAT is accessible in EW> various packages, including CL-USER. Maybe you're seeing a EW> package conflict?) RPG> The problem is not the FLEXI-STREAMS::EXTERNAL-FORMAT symbol, RPG> but the use of :external-format as an initarg, IIUC. See the RPG> (relatively) simple test case below for more details. EW> It'd be nice if you could provide a simple test case which EW> provokes this error message, so we can see what's really going EW> on. RPG> OK. Understand that I don't really know how flexi-streams RPG> are supposed to work --- I just stumbled into this because I RPG> use BEIRC, which uses CL-IRC, which (newly) uses RPG> FLEXI-STREAMS. So I'm pretty far from understanding things, RPG> and some of the things that I give may not be bugs; they may RPG> be me using the package wrong. RPG> My first shot at causing an error is the following: RPG> (defpackage :flexi-test (:use :common-lisp :flex)) RPG> (in-package :flexi-test) (setf foo (open "/tmp/foo" RPG> :direction :io)) (setf bar (make-flexi-stream foo)) Error: RPG> :DEFAULT is not known to be a name for an external format. RPG> Restart actions (select using :continue): RPG> 0: Return to Top Level (an "abort" restart). 1: Abort RPG> entirely from this (lisp) process. RPG> [1] FLEXI-TEST(18): :bt Evaluation stack: RPG> FLEX::NORMALIZE-EXTERNAL-FORMAT-NAME <- RPG> FLEX::MAKE-EXTERNAL-FORMAT% <- MAKE-EXTERNAL-FORMAT <- RPG> FLEX::MAYBE-CONVERT-EXTERNAL-FORMAT <- (METHOD RPG> INITIALIZE-INSTANCE :AFTER ...) <- (:INTERNAL RPG> (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE RPG> (CLASS)) <- (METHOD MAKE-INSTANCE (SYMBOL)) <- RPG> MAKE-FLEXI-STREAM <- [... EXCL::%EVAL ] <- LET* <- RPG> [... EXCL::%EVAL ] <- EVAL <- RPG> TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP <- RPG> TPL:START-INTERACTIVE-TOP-LEVEL RPG> [I'm honestly not sure where the :default value comes from, RPG> and it may be that I am simply committing an error here and RPG> that the :external-format keyword is mandatory, rather than RPG> optional, as I have treated it.] RPG> So I tried to do what cl-irc does: RPG> (defun external-format-fixup (format) RPG> (let ((new-format (copy-list format))) RPG> (setf (getf (cdr new-format) :eol-style) :crlf) RPG> new-format)) RPG> (defun mock-connect (&key (server "irc.freenode.net") RPG> (port 6667) RPG> (connection-type RPG> 'connection) RPG> (logging-stream t)) RPG> (let* ((stream (socket-connect server port)) RPG> (connection (make-connection RPG> :connection-type connection-type RPG> :network-stream stream RPG> :client-stream RPG> logging-stream RPG> :server-name server))) RPG> (values connection stream))) RPG> (defun make-connection (&key (connection-type 'connection) RPG> (user nil) (password nil) RPG> (server-name "") (server-port RPG> nil) (network-stream nil) RPG> (outgoing-external-format RPG> *default-outgoing-external-format*) RPG> (client-stream t) (hooks nil)) RPG> (let ((output-stream (flexi-streams:make-flexi-stream RPG> network-stream :element-type RPG> 'character :external-format RPG> (external-format-fixup RPG> outgoing-external-format)))) RPG> output-stream)) RPG> ;; overlooked this dependency.... RPG> (asdf:oos 'asdf:load-op :trivial-sockets) (defun RPG> socket-connect (server port) RPG> "Create a socket connected to `server':`port' and return RPG> stream for it." (trivial-sockets:open-stream server port RPG> :element-type '(unsigned-byte 8))) RPG> (setf *default-outgoing-external-format* '(:utf8)) That gets RPG> me the error: FLEXI-TEST(62): (mock-connect) Error: RPG> `:EOL-STYLE' does not name an external-format. RPG> [condition type: NO-EXTERNAL-FORMAT-ERROR] RPG> Restart actions (select using :continue): RPG> 0: Return to Top Level (an "abort" restart). 1: Abort RPG> entirely from this (lisp) process. RPG> [1] FLEXI-TEST(63): :bt Evaluation stack: RPG> (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE ...) <- RPG> (:INTERNAL (:EFFECTIVE-METHOD 2 NIL ...) 0) <- (:INTERNAL RPG> (:EFFECTIVE-METHOD 2 T ...) 0) <- (METHOD RPG> INITIALIZE-INSTANCE (STANDARD-OBJECT)) <- (:INTERNAL RPG> (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE RPG> (CLASS)) <- (METHOD MAKE-INSTANCE (SYMBOL)) <- RPG> MAKE-FLEXI-STREAM <- LET <- MAKE-CONNECTION <- LET* <- RPG> MOCK-CONNECT <- EVAL <- TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP RPG> <- TPL:START-INTERACTIVE-TOP-LEVEL RPG> The error happens when making the flexi-io-stream class. It RPG> triggers this ACL :before method that checks the value of the RPG> stream-external-format, and which seems to be seeing the RPG> '(:utf8 :eol-style :crlf) value that cl-irc is stuffing into RPG> the flexi-stream constructor. RPG> ACL has a method that evidently checks the assignment to RPG> stream-external-format for every stream to make sure that it RPG> is a legitimate excl-external-format, IIUC. A flexi-stream RPG> ISA stream, so somehow this is triggered. Your email seems RPG> to suggest that you did NOT expect the flexi-stream-value to RPG> become the stream-external-format, right? But this is the RPG> make-instance call: RPG> ((METHOD MAKE-INSTANCE (CLASS)) #<STANDARD-CLASS RPG> FLEXI-IO-STREAM> :STREAM RPG> #<MULTIVALENT stream socket connected from RPG> #192.168.1.18/41332 to RPG> kornbluth.freenode.net/6667 @ #x71dc7412> RPG> :ELEMENT-TYPE CHARACTER :EXTERNAL-FORMAT (:UTF8 :EOL-STYLE RPG> :CRLF)) RPG> So even though flexi-streams::external-format is not RPG> exported, flexi-streams is using the :external-format keyword RPG> in its make-instance call, and that is triggering the ACL RPG> code. Would it be enough to fix things to change the RPG> :initarg to ":flexi-stream-external-format"? That seems RPG> cumbersome, though. I don't know another way to keep ACL RPG> from grabbing up the :external-format keyword arg, though... RPG> But again, I have not thought deeply about this. >>>>> "EW" == Edi Weitz <edi(a)agharta.de> writes: EW> I think you're basically right and what happens is this: CL-USER> (defclass foo (excl::fundamental-character-stream) EW> ((bar :initarg :external-format EW> :initform (error "No external EW> format.")))) EW> #<STANDARD-CLASS FOO> CL-USER> (make-instance 'foo) EW> #<FOO @ #x2122c9aa> CL-USER> (slot-value * 'bar) EW> :DEFAULT EW> So, it looks like there's some :AROUND method within AllegroCL EW> which modifies the initargs before the FOO instance is EW> initialized. I couldn't find a place in their documentation EW> where they explicitely talk about the :EXTERNAL-FORMAT EW> initarg, and I'm hesitant to rename it in FLEXI-STREAMS EW> because it is the right name, isn't it? EW> At least, before I do that, it'd be nice if you could ask EW> Franz about their take. EW> Cheers, Edi. -- Robert P. Goldman Senior Scientist Smart Information Flow Technologies (d/b/a SIFT, LLC) 211 N. First St., Suite 300 Minneapolis, MN 55401 Voice: (612) 384-3454 Email: rpgoldman(a)SIFT.info

1 0

[flexi-streams-devel] Re: note on flexi-streams
by Edi Weitz 13 Jun '06

13 Jun '06

Hi! So, I'm back from vacation now... :) I'm Ccing the mailing list and I'd prefer if we could continue the discussion there. Thanks. On Thu, 1 Jun 2006 14:35:26 -0500, "Robert P. Goldman" <rpgoldman(a)sift.info> wrote: > I just pulled a copy of flexi-streams, using asdf-install, and had a > couple of minor problems: > > 1. The files have emacs mode lines which give a package name that > seems like it might be an old nickname for flexi-streams (effbf). > The in-package forms have the real package name. I don't know what > this does to slime, but it badly confuses ACL's emacs-lisp > interface, which attends to the mode line in preference to the > in-package form. Thanks, I've fixed that and made a new release (0.5.6). > 2. Making a flexible stream with an external format blows up for me > on ACL. E.g., cl-irc tries to make a flexi-io-stream with external > format (:utf-8 :eof-type :crlf) > > This value is stuffed into the flexi-stream slot EXTERNAL-FORMAT. > Unfortunately, IIUC, this also triggers an Allegro-specific method > as follows: > > (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE (T STREAM)) > > This method checks to make sure that the value you are setting as a > stream-external-format is an excl:external-format object. In this > case, it clearly isn't, so the :before method hurls an error. > > I believe that this error is triggered because a flexi-io-stream ISA > fundamental-binary-input-stream which ISA stream. > > I confess I'm not really sure what to do about this problem.... Hmm, at first glance I'd say that this is either an AllegroCL bug or it is not related to FLEXI-STREAMS at all. The slot you're talking about above is called FLEXI-STREAMS::EXTERNAL-FORMAT, its name is not exported from the package. I can't see why setting the value of this slot should trigger an Allegro-specific method. (Note that on AllegroCL EXCL:EXTERNAL-FORMAT is accessible in various packages, including CL-USER. Maybe you're seeing a package conflict?) It'd be nice if you could provide a simple test case which provokes this error message, so we can see what's really going on. > Here's an error backtrace: > > [1c] IRC(100): (setf str > (flexi-streams:make-flexi-stream > network-stream > :element-type 'character > :external-format > (external-format-fixup > *default-outgoing-external-format*) > )) > Error: `:EOL-STYLE' does not name an external-format. > [condition type: NO-EXTERNAL-FORMAT-ERROR] > > Restart actions (select using :continue): > 0: Return to Debug Level 1 (an "abort" restart). > 1: Try calling it again > 2: Return to Top Level (an "abort" restart). > 3: Abort entirely from this (lisp) process. > [2] IRC(101): :bt > Evaluation stack: > > (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE ...) <- > (:INTERNAL (:EFFECTIVE-METHOD 2 NIL ...) 0) <- > (:INTERNAL (:EFFECTIVE-METHOD 2 T ...) 0) <- > (METHOD INITIALIZE-INSTANCE (STANDARD-OBJECT)) <- > (:INTERNAL (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE > (CLASS)) <- > (METHOD MAKE-INSTANCE (SYMBOL)) <- FLEX:MAKE-FLEXI-STREAM <- LET* <- > [... EXCL::%EVAL ] <- EVAL <- CERROR <- (METHOD NO-APPLICABLE-METHOD > (T)) <- > FLEX::TRANSLATE-CHAR <- > (METHOD EXCL:STREAM-WRITE-CHAR (FLEX:FLEXI-OUTPUT-STREAM T)) <- > (METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE > (FLEX:FLEXI-OUTPUT-STREAM T T ...)) <- > [... TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE ] <- > (METHOD EXCL:STREAM-WRITE-SEQUENCE > (TRIVIAL-GRAY-STREAMS:TRIVIAL-GRAY-STREAM-MIXIN T)) <- > (METHOD SEND-IRC-MESSAGE (CONNECTION T)) <- [... SEND-IRC-MESSAGE ] > <- > (METHOD NICK (CONNECTION STRING)) <- CONNECT <- BEIRC::COM-CONNECT > <- > (METHOD CLIM:EXECUTE-FRAME-COMMAND (CLIM:APPLICATION-FRAME T)) <- > (METHOD CLIM:ADOPT-FRAME :AFTER ...) <- > (:INTERNAL (:EFFECTIVE-METHOD 2 NIL ...) 0) <- > (METHOD CLIM:RUN-FRAME-TOP-LEVEL :AROUND ...) <- > (:INTERNAL (:EFFECTIVE-METHOD 1 T ...) 0) <- > [... CLIM:RUN-FRAME-TOP-LEVEL ] <- > (:INTERNAL BEIRC:BEIRC 0) <- BEIRC:BEIRC <- [... EXCL::%EVAL ] <- > EVAL <- > TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP <- > TPL:START-INTERACTIVE-TOP-LEVEL Cheers, Edi.

2 2