Attached is an implementation for SBCL. make-static-vector only allocates instances of (simple-array (unsigned-byte 8) (*)) but that can be extended to other unboxed arrays. The current sharable vector interface doesn't work very well because it doesn't allow me for instance to pin a list of vectors, but only a single vector at a time. Also, I'd prefer to avoid pinning at all given how it interferes with the GC.
Stelian Ionescu stelian.ionescu-zeus@poste.it writes:
Attached is an implementation for SBCL. make-static-vector only allocates instances of (simple-array (unsigned-byte 8) (*)) but that can be extended to other unboxed arrays. The current sharable vector interface doesn't work very well because it doesn't allow me for instance to pin a list of vectors, but only a single vector at a time. Also, I'd prefer to avoid pinning at all given how it interferes with the GC.
Thanks Stelian!
This would be really useful as the pinning business with shareable-byte-vectors is a real nuisance.
If the current shareable-byte-vector interface is going to be reworked can I suggest that we allow (at least) simple-array single-float/double-float as well? Why not just allow all unboxed array types the underlying Lisp supports? . . . Or a limited sensible subset?
Having portable unboxed C-sharable float arrays would be great for our numeric work.
I should be very happy to make the Allegro CL interface.
On Thu, 2009-05-07 at 11:00 +0900, John Fremlin wrote:
Stelian Ionescu stelian.ionescu-zeus@poste.it writes:
Attached is an implementation for SBCL. make-static-vector only allocates instances of (simple-array (unsigned-byte 8) (*)) but that can be extended to other unboxed arrays. The current sharable vector interface doesn't work very well because it doesn't allow me for instance to pin a list of vectors, but only a single vector at a time. Also, I'd prefer to avoid pinning at all given how it interferes with the GC.
Thanks Stelian!
This would be really useful as the pinning business with shareable-byte-vectors is a real nuisance.
If the current shareable-byte-vector interface is going to be reworked can I suggest that we allow (at least) simple-array single-float/double-float as well? Why not just allow all unboxed array
That can be easily done.
types the underlying Lisp supports? . . . Or a limited sensible subset?
I think that we should allow all types supported by the underlying implementation but specify in the docs that very few are portable: (probably, I haven't yet checked) only bit, [un]signed-byte-{8,16,32,64} and single-float are portable
Having portable unboxed C-sharable float arrays would be great for our numeric work.
Now we only need to see if the trick of "manually" creating unboxed arrays works on other implementations: CMUCL, CCL, Clisp, ECL, ... Allegro and Lispworks already support this with make-array's :ALLOCATION key arg
I should be very happy to make the Allegro CL interface.
Thanks. See if Allegro has native functions for bit-bashing, otherwise implement {fill,copy}-foreign-memory in terms of memset/memcpy.
Stelian Ionescu stelian.ionescu-zeus@poste.it writes: [...]
This would be really useful as the pinning business with shareable-byte-vectors is a real nuisance.
If the current shareable-byte-vector interface is going to be reworked can I suggest that we allow (at least) simple-array single-float/double-float as well? Why not just allow all unboxed array
That can be easily done.
Indeed.
A better alternative to shareable byte-vectors is absolutely a good idea.
Can you post the revised proposed interface?
Can ClozureCL handle it, or does it need to be modified?
types the underlying Lisp supports? . . . Or a limited sensible subset?
I think that we should allow all types supported by the underlying implementation but specify in the docs that very few are portable: (probably, I haven't yet checked) only bit, [un]signed-byte-{8,16,32,64} and single-float are portable
I guess double-float is more portable than [un]signed-byte-64 (not sure).
Having portable unboxed C-sharable float arrays would be great for our numeric work.
Now we only need to see if the trick of "manually" creating unboxed arrays works on other implementations: CMUCL, CCL, Clisp, ECL, ... Allegro and Lispworks already support this with make-array's :ALLOCATION key arg
Yes, though note that Allegro sometimes annoyingly makes a non-simple array with certain allocations.
I should be very happy to make the Allegro CL interface.
Thanks. See if Allegro has native functions for bit-bashing, otherwise implement {fill,copy}-foreign-memory in terms of memset/memcpy.
Why not just use memset and memcpy always? I rather expect that the C library version are much better than things implemented by most Lisps (Allegro especially seems to be stuck in the 1990s when it comes to CPU memory/architectures).
On Thu, 07 May 2009 09:48:08 +0200, Stelian Ionescu wrote:
On Thu, 2009-05-07 at 11:00 +0900, John Fremlin wrote:
Stelian Ionescu stelian.ionescu-zeus@poste.it writes:
Attached is an implementation for SBCL. make-static-vector only allocates instances of (simple-array (unsigned-byte 8) (*)) but that can be extended to other unboxed arrays. The current sharable vector interface doesn't work very well because it doesn't allow me for instance to pin a list of vectors, but only a single vector at a time. Also, I'd prefer to avoid pinning at all given how it interferes with the GC.
Thanks Stelian!
This would be really useful as the pinning business with shareable-byte-vectors is a real nuisance.
If the current shareable-byte-vector interface is going to be reworked can I suggest that we allow (at least) simple-array single-float/double-float as well? Why not just allow all unboxed array
That can be easily done.
types the underlying Lisp supports? . . . Or a limited sensible subset?
I think that we should allow all types supported by the underlying implementation but specify in the docs that very few are portable: (probably, I haven't yet checked) only bit, [un]signed-byte-{8,16,32,64} and single-float are portable
Having portable unboxed C-sharable float arrays would be great for our numeric work.
Now we only need to see if the trick of "manually" creating unboxed arrays works on other implementations: CMUCL, CCL, Clisp, ECL, ... Allegro and Lispworks already support this with make-array's :ALLOCATION key arg
I should be very happy to make the Allegro CL interface.
Thanks. See if Allegro has native functions for bit-bashing, otherwise implement {fill,copy}-foreign-memory in terms of memset/memcpy.
Thanks Stefan,
Very neat work. This would indeed help a lot with numeric work.
Question: is the need to manually free the object a must? Could we perhaps have an argument to make-static-vector that tells it to register a finalizer when needed?
Tamas
On Fri, 2009-05-08 at 19:52 +0000, Tamas K Papp wrote:
On Thu, 07 May 2009 09:48:08 +0200, Stelian Ionescu wrote:
On Thu, 2009-05-07 at 11:00 +0900, John Fremlin wrote:
Stelian Ionescu stelian.ionescu-zeus@poste.it writes:
Attached is an implementation for SBCL. make-static-vector only allocates instances of (simple-array (unsigned-byte 8) (*)) but that can be extended to other unboxed arrays. The current sharable vector interface doesn't work very well because it doesn't allow me for instance to pin a list of vectors, but only a single vector at a time. Also, I'd prefer to avoid pinning at all given how it interferes with the GC.
Thanks Stelian!
This would be really useful as the pinning business with shareable-byte-vectors is a real nuisance.
If the current shareable-byte-vector interface is going to be reworked can I suggest that we allow (at least) simple-array single-float/double-float as well? Why not just allow all unboxed array
That can be easily done.
types the underlying Lisp supports? . . . Or a limited sensible subset?
I think that we should allow all types supported by the underlying implementation but specify in the docs that very few are portable: (probably, I haven't yet checked) only bit, [un]signed-byte-{8,16,32,64} and single-float are portable
Having portable unboxed C-sharable float arrays would be great for our numeric work.
Now we only need to see if the trick of "manually" creating unboxed arrays works on other implementations: CMUCL, CCL, Clisp, ECL, ... Allegro and Lispworks already support this with make-array's :ALLOCATION key arg
I should be very happy to make the Allegro CL interface.
Thanks. See if Allegro has native functions for bit-bashing, otherwise implement {fill,copy}-foreign-memory in terms of memset/memcpy.
Thanks Stefan,
Very neat work. This would indeed help a lot with numeric work.
Question: is the need to manually free the object a must? Could we perhaps have an argument to make-static-vector that tells it to register a finalizer when needed?
Yes. Given that SBCL has a conservative GC on x86 and x86_64 you have no guarantee that a finalizer will ever be executed. Also, I'm not sure how it would interact with the fact that the entire array(contents and header) is allocated in foreign memory. It's better not to risk and if some day we'll be sure that it's unnecessary on all implementations, we could safely deprecate free-static-vector and use finalizers.
Stelian Ionescu stelian.ionescu-zeus@poste.it writes:
Yes. Given that SBCL has a conservative GC on x86 and x86_64 you have no guarantee that a finalizer will ever be executed. Also, I'm not sure how it would interact with the fact that the entire array(contents and header) is allocated in foreign memory. It's better not to risk and if some day we'll be sure that it's unnecessary on all implementations, we could safely deprecate free-static-vector and use finalizers.
It would certainly be very nice to have a portable way of letting the GC dispose of garbage but stop it from moving buffers around.
Allegro already supports this just fine, for example.
-
John Fremlin -
Stelian Ionescu -
Tamas K Papp