Does anybody know of a good way to serialize lisp objects into java byte arrays? I've got a method that almost works, but if anybody has a better/canonical way of doing this I'd love to know!
I've been trying to use cl-store, outputting to a stream I create with sys::%make-byte-array-output-stream, and then getting the byte array with sys::%get-output-stream-bytes. I've only been using abcl for a few days, and I found those functions by grepping through the sources, so I apologize if I'm inviting trouble by messing around with functions you never meant users to play with.
Anyway, the output works fine. But I can't find the equivalent of sys::%make-byte-array-output-stream to create an input stream, so I attempted to make my own
(defun make-byte-array-input-stream (bytes) (let ((bis (jnew (jconstructor "java.io.ByteArrayInputStream" "[B") bytes))) (jnew (jconstructor "org.armedbear.lisp.Stream" "java.io.InputStream") bis)))
This almost seems to work, but if I try to use it I get a java exception and a (very long) java stack trace. I put a copy of the stack trace at the end of this message. Here is an expression which will produce the problem in abcl-18.1
(let ((outbytes (with-open-stream (stream (sys::%make-byte-array-output-stream)) (write-byte 101 stream) (sys::%get-output-stream-bytes stream)))) ;; now open the byte array as an input stream (with-open-stream (stream (make-byte-array-input-stream outbytes)) ;; all works fine until here (read-byte stream)))
I don't need to read the stream, asking for the element type of the input stream is enough to cause trouble ...
(let ((outbytes (with-open-stream (stream (sys::%make-byte-array-output-stream)) (write-byte 101 stream) (sys::%get-output-stream-bytes stream)))) (with-open-stream (stream (make-byte-array-input-stream outbytes)) (stream-element-type stream)))
I can fix the problem and make my code work, but I have to change Stream.java to make it work. I added
elementType = UNSIGNED_BYTE_8;
to the initAsBinaryInputStream(InputStream in) method. I did this because the stack trace suggested that the problem was stream_element_type.execute() was returning null.
Anyway, that fixes the problem, and I'm happily serializing/deserializing lisp objects to a Berkeley DB ... so I guess I'm happy! But I don't know how rational that fix is, because I don't really know how Stream is supposed to work. I suspect that my 'fix' probably breaks something else that I'm just not using yet.
Any ideas?
Thanks,
-david k.
Here's the first few lines of the stack trace ...
CL-USER(2): Debugger invoked on condition of type SIMPLE-TYPE-ERROR: java.lang.NullPointerException at org.armedbear.lisp.Primitives$49.execute(Primitives.java:795) at org.armedbear.lisp.Symbol.execute(Symbol.java:781) at org.armedbear.lisp.LispThread.execute(LispThread.java:579) at org.armedbear.lisp.print_8.execute(print.lisp:127) at org.armedbear.lisp.Symbol.execute(Symbol.java:781) at org.armedbear.lisp.LispThread.execute(LispThread.java:579) at org.armedbear.lisp.print_14.execute(print.lisp:279)
... (and it ends with)
at org.armedbear.lisp.AutoloadedFunctionProxy.execute(AutoloadedFunctionProxy.java:139) at org.armedbear.lisp.Symbol.execute(Symbol.java:760) at org.armedbear.lisp.LispThread.execute(LispThread.java:546) at org.armedbear.lisp.top_level_51.execute(top-level.lisp:415) at org.armedbear.lisp.AutoloadedFunctionProxy.execute(AutoloadedFunctionProxy.java:139) at org.armedbear.lisp.LispThread.execute(LispThread.java:546) at org.armedbear.lisp.Interpreter.run(Interpreter.java:310) at org.armedbear.lisp.Main$1.run(Main.java:50) at java.lang.Thread.run(Thread.java:613) READ-BYTE: unsupported element type Debugger invoked on condition of type ERROR: Caught java.lang.NullPointerException. Restarts: 0: TOP-LEVEL Return to top level.
On Wed, Mar 17, 2010 at 12:06 AM, David Kirkman dkirkman@ucsd.edu wrote:
Does anybody know of a good way to serialize lisp objects into java byte arrays? I've got a method that almost works, but if anybody has a better/canonical way of doing this I'd love to know!
I've been trying to use cl-store, outputting to a stream I create with sys::%make-byte-array-output-stream, and then getting the byte array with sys::%get-output-stream-bytes. I've only been using abcl for a few days, and I found those functions by grepping through the sources, so I apologize if I'm inviting trouble by messing around with functions you never meant users to play with.
Anyway, the output works fine. But I can't find the equivalent of sys::%make-byte-array-output-stream to create an input stream, so I attempted to make my own
(defun make-byte-array-input-stream (bytes) (let ((bis (jnew (jconstructor "java.io.ByteArrayInputStream" "[B") bytes))) (jnew (jconstructor "org.armedbear.lisp.Stream" "java.io.InputStream") bis)))
This almost seems to work, but if I try to use it I get a java exception and a (very long) java stack trace. I put a copy of the stack trace at the end of this message. Here is an expression which will produce the problem in abcl-18.1
(let ((outbytes (with-open-stream (stream (sys::%make-byte-array-output-stream)) (write-byte 101 stream) (sys::%get-output-stream-bytes stream)))) ;; now open the byte array as an input stream (with-open-stream (stream (make-byte-array-input-stream outbytes)) ;; all works fine until here (read-byte stream)))
I don't need to read the stream, asking for the element type of the input stream is enough to cause trouble ...
(let ((outbytes (with-open-stream (stream (sys::%make-byte-array-output-stream)) (write-byte 101 stream) (sys::%get-output-stream-bytes stream)))) (with-open-stream (stream (make-byte-array-input-stream outbytes)) (stream-element-type stream)))
I can fix the problem and make my code work, but I have to change Stream.java to make it work. I added
elementType = UNSIGNED_BYTE_8;
to the initAsBinaryInputStream(InputStream in) method. I did this because the stack trace suggested that the problem was stream_element_type.execute() was returning null.
Anyway, that fixes the problem, and I'm happily serializing/deserializing lisp objects to a Berkeley DB ... so I guess I'm happy! But I don't know how rational that fix is, because I don't really know how Stream is supposed to work. I suspect that my 'fix' probably breaks something else that I'm just not using yet.
Any ideas?
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Alessio
Thanks,
-david k.
Here's the first few lines of the stack trace ...
CL-USER(2): Debugger invoked on condition of type SIMPLE-TYPE-ERROR: java.lang.NullPointerException at org.armedbear.lisp.Primitives$49.execute(Primitives.java:795) at org.armedbear.lisp.Symbol.execute(Symbol.java:781) at org.armedbear.lisp.LispThread.execute(LispThread.java:579) at org.armedbear.lisp.print_8.execute(print.lisp:127) at org.armedbear.lisp.Symbol.execute(Symbol.java:781) at org.armedbear.lisp.LispThread.execute(LispThread.java:579) at org.armedbear.lisp.print_14.execute(print.lisp:279)
... (and it ends with)
at org.armedbear.lisp.AutoloadedFunctionProxy.execute(AutoloadedFunctionProxy.java:139) at org.armedbear.lisp.Symbol.execute(Symbol.java:760) at org.armedbear.lisp.LispThread.execute(LispThread.java:546) at org.armedbear.lisp.top_level_51.execute(top-level.lisp:415) at org.armedbear.lisp.AutoloadedFunctionProxy.execute(AutoloadedFunctionProxy.java:139) at org.armedbear.lisp.LispThread.execute(LispThread.java:546) at org.armedbear.lisp.Interpreter.run(Interpreter.java:310) at org.armedbear.lisp.Main$1.run(Main.java:50) at java.lang.Thread.run(Thread.java:613) READ-BYTE: unsupported element type Debugger invoked on condition of type ERROR: Caught java.lang.NullPointerException. Restarts: 0: TOP-LEVEL Return to top level.
armedbear-devel mailing list armedbear-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Bye, Alessio
-- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
armedbear-devel mailing list armedbear-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Committed as r12557.
A.
On Wed, Mar 17, 2010 at 9:55 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Committed as r12557.
A.
Thanks for implementing the inverse function so quickly!
I ended up doing a bit more hacking last night. When I tried to serialize structures that contain raw java objects, I realized that I need a way to convert lisp streams into java streams so that I can use the java serialization system. I ended up writing:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return new OutputStream() { public void write(int b) { ((org.armedbear.lisp.Stream) stream)._writeByte(b); } }; } else { throw new RuntimeException("argument not an abcl stream"); } }
and another version for InputStream. Should something like these also go in as primitives?
-david k.
On Wed, Mar 17, 2010 at 7:36 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 9:55 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Committed as r12557.
A.
Thanks for implementing the inverse function so quickly!
I ended up doing a bit more hacking last night. When I tried to serialize structures that contain raw java objects, I realized that I need a way to convert lisp streams into java streams so that I can use the java serialization system. I ended up writing:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return new OutputStream() { public void write(int b) { ((org.armedbear.lisp.Stream) stream)._writeByte(b); } }; } else { throw new RuntimeException("argument not an abcl stream"); } }
and another version for InputStream. Should something like these also go in as primitives?
For binary input/output streams only, I think accessing the Java streams wrapped by the Lisp stream is a better solution: no extra objects are allocated and no API mismatch is possible. Unfortunately those are private fields right now, but a couple of getters are not hard to add ;)
so, your outputStreamFromABCLStream would become:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); } else { throw new RuntimeException("argument not an abcl stream"); } }
or, if you don't mind getting a ClassCastException instead, simply
public static OutputStream outputStreamFromABCLStream(final LispObject stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); }
what do you think?
Alessio
On Wed, Mar 17, 2010 at 11:51 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 7:36 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 9:55 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Committed as r12557.
A.
Thanks for implementing the inverse function so quickly!
I ended up doing a bit more hacking last night. When I tried to serialize structures that contain raw java objects, I realized that I need a way to convert lisp streams into java streams so that I can use the java serialization system. I ended up writing:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return new OutputStream() { public void write(int b) { ((org.armedbear.lisp.Stream) stream)._writeByte(b); } }; } else { throw new RuntimeException("argument not an abcl stream"); } }
and another version for InputStream. Should something like these also go in as primitives?
For binary input/output streams only, I think accessing the Java streams wrapped by the Lisp stream is a better solution: no extra objects are allocated and no API mismatch is possible. Unfortunately those are private fields right now, but a couple of getters are not hard to add ;)
so, your outputStreamFromABCLStream would become:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); } else { throw new RuntimeException("argument not an abcl stream"); } }
or, if you don't mind getting a ClassCastException instead, simply
public static OutputStream outputStreamFromABCLStream(final LispObject stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); }
what do you think?
I think that's a much better solution! I don't mind ClassCastExceptions, my code had that structure because it started as something copy/pasted out of Stream.java. And I was using _readByte and _writeByte just because I didn't want to have to make changes to Stream.java and end up with my own version of abcl.
-david k.
On Wed, Mar 17, 2010 at 8:10 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 11:51 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 7:36 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 9:55 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
> I wrote sys::%make-byte-array-output-stream in order to make the > runtime compiler capable of generating bytecode without using > temporary files; it wasn't meant to be used by the ABCL user, and as > such it's not very polished (for example, the type of a > byte-array-output-stream is simply STREAM). In any case, it's defined > in the ByteArrayOutputStream Java class, and as you correctly noted, > it explicitly sets the element type to (unsigned-byte 8). As a quick > and dirty solution, you could more or less copy-paste that class and > replace "output" with "input" :) (as well as update Autoload.java to > make ABCL know of the new primitives). > > We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Committed as r12557.
A.
Thanks for implementing the inverse function so quickly!
I ended up doing a bit more hacking last night. When I tried to serialize structures that contain raw java objects, I realized that I need a way to convert lisp streams into java streams so that I can use the java serialization system. I ended up writing:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return new OutputStream() { public void write(int b) { ((org.armedbear.lisp.Stream) stream)._writeByte(b); } }; } else { throw new RuntimeException("argument not an abcl stream"); } }
and another version for InputStream. Should something like these also go in as primitives?
For binary input/output streams only, I think accessing the Java streams wrapped by the Lisp stream is a better solution: no extra objects are allocated and no API mismatch is possible. Unfortunately those are private fields right now, but a couple of getters are not hard to add ;)
so, your outputStreamFromABCLStream would become:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); } else { throw new RuntimeException("argument not an abcl stream"); } }
or, if you don't mind getting a ClassCastException instead, simply
public static OutputStream outputStreamFromABCLStream(final LispObject stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); }
what do you think?
I think that's a much better solution! I don't mind ClassCastExceptions, my code had that structure because it started as something copy/pasted out of Stream.java. And I was using _readByte and _writeByte just because I didn't want to have to make changes to Stream.java and end up with my own version of abcl.
Ok. I committed the modified Stream with accessors for the underlying Java streams.
Bye, Alessio
On Wed, Mar 17, 2010 at 12:15 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 8:10 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 11:51 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 7:36 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 9:55 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote: > On 3/17/10 5:04 PM, Alessio Stalla wrote: > […] > >> I wrote sys::%make-byte-array-output-stream in order to make the >> runtime compiler capable of generating bytecode without using >> temporary files; it wasn't meant to be used by the ABCL user, and as >> such it's not very polished (for example, the type of a >> byte-array-output-stream is simply STREAM). In any case, it's defined >> in the ByteArrayOutputStream Java class, and as you correctly noted, >> it explicitly sets the element type to (unsigned-byte 8). As a quick >> and dirty solution, you could more or less copy-paste that class and >> replace "output" with "input" :) (as well as update Autoload.java to >> make ABCL know of the new primitives). >> >> We should probably polish it a bit and release it as an extension. > > Well, at least we should quickly whip out the inverse version so David > doesn't have to rely on hacking Stream.java (although I'm not sure when > telling Stream that initAsBinaryStream() means a format of unsigned 8bit > bytes will fail). > > I think we can commit to the interface you've implemented in > ByteArrayOutputStream, as those are the basic necessary operations. > > Or maybe I am not seeing what potential problems there would be with > this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
Committed as r12557.
A.
Thanks for implementing the inverse function so quickly!
I ended up doing a bit more hacking last night. When I tried to serialize structures that contain raw java objects, I realized that I need a way to convert lisp streams into java streams so that I can use the java serialization system. I ended up writing:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return new OutputStream() { public void write(int b) { ((org.armedbear.lisp.Stream) stream)._writeByte(b); } }; } else { throw new RuntimeException("argument not an abcl stream"); } }
and another version for InputStream. Should something like these also go in as primitives?
For binary input/output streams only, I think accessing the Java streams wrapped by the Lisp stream is a better solution: no extra objects are allocated and no API mismatch is possible. Unfortunately those are private fields right now, but a couple of getters are not hard to add ;)
so, your outputStreamFromABCLStream would become:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); } else { throw new RuntimeException("argument not an abcl stream"); } }
or, if you don't mind getting a ClassCastException instead, simply
public static OutputStream outputStreamFromABCLStream(final LispObject stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); }
what do you think?
I think that's a much better solution! I don't mind ClassCastExceptions, my code had that structure because it started as something copy/pasted out of Stream.java. And I was using _readByte and _writeByte just because I didn't want to have to make changes to Stream.java and end up with my own version of abcl.
Ok. I committed the modified Stream with accessors for the underlying Java streams.
You guys are awesome. But I can't build trunk now. It seems that getInputStream conflicts with methods already defined in TwoWayStream and EchoStream. How about getBinaryInputStream?
-david k.
On Wed, Mar 17, 2010 at 8:27 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 12:15 PM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 8:10 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 11:51 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 7:36 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 9:55 AM, Alessio Stalla alessiostalla@gmail.com wrote:
On Wed, Mar 17, 2010 at 5:32 PM, Alessio Stalla alessiostalla@gmail.com wrote: > On Wed, Mar 17, 2010 at 5:22 PM, Mark Evenson evenson@panix.com wrote: >> On 3/17/10 5:04 PM, Alessio Stalla wrote: >> […] >> >>> I wrote sys::%make-byte-array-output-stream in order to make the >>> runtime compiler capable of generating bytecode without using >>> temporary files; it wasn't meant to be used by the ABCL user, and as >>> such it's not very polished (for example, the type of a >>> byte-array-output-stream is simply STREAM). In any case, it's defined >>> in the ByteArrayOutputStream Java class, and as you correctly noted, >>> it explicitly sets the element type to (unsigned-byte 8). As a quick >>> and dirty solution, you could more or less copy-paste that class and >>> replace "output" with "input" :) (as well as update Autoload.java to >>> make ABCL know of the new primitives). >>> >>> We should probably polish it a bit and release it as an extension. >> >> Well, at least we should quickly whip out the inverse version so David >> doesn't have to rely on hacking Stream.java (although I'm not sure when >> telling Stream that initAsBinaryStream() means a format of unsigned 8bit >> bytes will fail). >> >> I think we can commit to the interface you've implemented in >> ByteArrayOutputStream, as those are the basic necessary operations. >> >> Or maybe I am not seeing what potential problems there would be with >> this as a short-term strategy, Alessio? > > No problem at all in the implementation per se; however, when we'll > make it an "official" extension, the interface will change (if > anything, symbols won't probably be in sys anymore and they won't be > prefixed with %). Since this functionality will have approximately 1 > user in the near future :), I think it's ok to commit a quick fix for > David. I'm working on it right now.
Committed as r12557.
A.
Thanks for implementing the inverse function so quickly!
I ended up doing a bit more hacking last night. When I tried to serialize structures that contain raw java objects, I realized that I need a way to convert lisp streams into java streams so that I can use the java serialization system. I ended up writing:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return new OutputStream() { public void write(int b) { ((org.armedbear.lisp.Stream) stream)._writeByte(b); } }; } else { throw new RuntimeException("argument not an abcl stream"); } }
and another version for InputStream. Should something like these also go in as primitives?
For binary input/output streams only, I think accessing the Java streams wrapped by the Lisp stream is a better solution: no extra objects are allocated and no API mismatch is possible. Unfortunately those are private fields right now, but a couple of getters are not hard to add ;)
so, your outputStreamFromABCLStream would become:
public static OutputStream outputStreamFromABCLStream(final LispObject stream) {
if (stream instanceof org.armedbear.lisp.Stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); } else { throw new RuntimeException("argument not an abcl stream"); } }
or, if you don't mind getting a ClassCastException instead, simply
public static OutputStream outputStreamFromABCLStream(final LispObject stream) { return ((org.armedbear.lisp.Stream) stream).getOutputStream(); }
what do you think?
I think that's a much better solution! I don't mind ClassCastExceptions, my code had that structure because it started as something copy/pasted out of Stream.java. And I was using _readByte and _writeByte just because I didn't want to have to make changes to Stream.java and end up with my own version of abcl.
Ok. I committed the modified Stream with accessors for the underlying Java streams.
You guys are awesome. But I can't build trunk now. It seems that getInputStream conflicts with methods already defined in TwoWayStream and EchoStream. How about getBinaryInputStream?
Whoops. Sorry. I did recompile abcl to check, but forgot to remove the incremental build flag I usually use. I renamed all the methods to getWrappedXXX().
A.
On Wed, Mar 17, 2010 at 12:43 PM, Alessio Stalla alessiostalla@gmail.com wrote:
Whoops. Sorry. I did recompile abcl to check, but forgot to remove the incremental build flag I usually use. I renamed all the methods to getWrappedXXX().
Great. That works.
Is there documentation about that incremental build flag? Every time I change something I'm left with about a five minute build. I'm currently using the ant based build method described in the README.
-david k.
On 3/17/10 9:38 PM, David Kirkman wrote:
Is there documentation about that incremental build flag? Every time I change something I'm left with about a five minute build. I'm currently using the ant based build method described in the README.
If you are on trunk building with Ant create a "abcl.properties" file with abcl.build.incremental=true
to get the incremental build option.
The only current documentation exists in "abcl.properties.in". I acknowledge this as a TODO...
On Wed, Mar 17, 2010 at 9:38 PM, David Kirkman dkirkman@ucsd.edu wrote:
On Wed, Mar 17, 2010 at 12:43 PM, Alessio Stalla alessiostalla@gmail.com wrote:
Whoops. Sorry. I did recompile abcl to check, but forgot to remove the incremental build flag I usually use. I renamed all the methods to getWrappedXXX().
Great. That works.
Is there documentation about that incremental build flag? Every time I change something I'm left with about a five minute build. I'm currently using the ant based build method described in the README.
ant -Dabcl.build.incremental abcl.jar
of course, you can miss some compilation errors, like in my case.
Best, Alessio
On 3/17/10 5:32 PM, Alessio Stalla wrote:
On Wed, Mar 17, 2010 at 5:22 PM, Mark Evensonevenson@panix.com wrote:
On 3/17/10 5:04 PM, Alessio Stalla wrote: […]
I wrote sys::%make-byte-array-output-stream in order to make the runtime compiler capable of generating bytecode without using temporary files; it wasn't meant to be used by the ABCL user, and as such it's not very polished (for example, the type of a byte-array-output-stream is simply STREAM). In any case, it's defined in the ByteArrayOutputStream Java class, and as you correctly noted, it explicitly sets the element type to (unsigned-byte 8). As a quick and dirty solution, you could more or less copy-paste that class and replace "output" with "input" :) (as well as update Autoload.java to make ABCL know of the new primitives).
We should probably polish it a bit and release it as an extension.
Well, at least we should quickly whip out the inverse version so David doesn't have to rely on hacking Stream.java (although I'm not sure when telling Stream that initAsBinaryStream() means a format of unsigned 8bit bytes will fail).
I think we can commit to the interface you've implemented in ByteArrayOutputStream, as those are the basic necessary operations.
Or maybe I am not seeing what potential problems there would be with this as a short-term strategy, Alessio?
No problem at all in the implementation per se; however, when we'll make it an "official" extension, the interface will change (if anything, symbols won't probably be in sys anymore and they won't be prefixed with %). Since this functionality will have approximately 1 user in the near future :), I think it's ok to commit a quick fix for David. I'm working on it right now.
I'd like to promote the %make-byte-array-output-stream/%make-byte-array-input-stream functions to the EXTENSIONS package, as I seem to have needed them at least twice in different contexts since we introduced them.
@Alessio: are you satisfied with the interface as they stand?
@All: It looks like we just to have finish the insertion in the type hierarchy or should they continue to be just instances of STREAM? Not sure what to do here.
After finishing the type hierarchy, I would merely make a Lisp stub in EXTENSIONS for now for MAKE-BYTE-ARRAY-OUTPUT-STREAM, MAKE-BYTE-ARRAY-INPUT-STREAM, GET-OUTPUT-STREAM-BYTES external symbols.
armedbear-devel@common-lisp.net
-
Alessio Stalla -
David Kirkman -
Mark Evenson