Today, I spent a bit of time trying to figure out a way to efficiently compute message digest for Quicklisp. After confirming that java.security.MessageDigest implements the required digests, I noticed we didn't have a way to easily get at the underlying Java byte[] array in (SIMPLE-VECTOR (UNSIGNED-BYTE 8)). So, I patched the underlying Java code as attached, and wrote the following routine:
(defun sha-256 (path) (let ((buffer (make-array 8192 :element-type '(unsigned-byte 8)) (digest (jstatic "getInstance" "java.security.MessageDigest" "SHA-256"))) (with-open-file (in path :element-type '(unsigned-byte 8)) (loop :for bytes = (read-sequence buffer in) :while (plusp bytes) :do (jcall-raw "update" digest (make-immediate-object buffer) 0 bytes)) (jcall "digest" digest)))))
But then I noticed that MAKE-IMMEDIATE-OBJECT calls JOBJECT-LISP-VALUE which makes no conceptual sense. The doc for MAKE-IMMEDIATE-OBJECT reads "[a]ttempts to coerce a given Lisp object into a java-object" while the doc JOBJECT-LISP-VALUE reads "Attempts to coerce JAVA-OBJECT into a Lisp object". This seems wrong here, but I don't understand how exactly. I think we need to distinguish three types of values here: JAVA-OBJECT, LispObject, and the "raw" value of the Java. How should we really structure our Java FFI here?
With my patch, for an underlying lisp value of type (SIMPLE-VECTOR (UNSIGNED-BYTE 8)), both MAKE-IMMEDIATE-OBJECT and JOBJECT-LISP-OBJECT return the byte[] array. I would think that MAKE-IMMEDIATE-OBJECT should just do this, which JOBJECT-LISP-OBJECT should .... return an error if its argument isn't of type JAVA-OBJECT? I don't know. I am confused.
Can someone help me understand the right way forward here?
I've always disliked make-immediate-object. Really it serves the purpose of disambiguating a few overloaded values in lisp. I'd like to deprecate it in favor of specific constants for java-true, java-false and null-pointer (the latter is already available as function).
It seems dangerous to assume that one can get some java array as the underlying object below a lisp array, what with all the hair that lisp arrays can have. Better to allocate your buffer with (jnew-array "byte" 8192) and not get into this game. Or have a documented policy of how java objects correspond to lisp objects and promise to support it forever.
2c, -Alan
On Fri, Jun 10, 2011 at 11:42 AM, Mark Evenson evenson@panix.com wrote:
Today, I spent a bit of time trying to figure out a way to efficiently compute message digest for Quicklisp. After confirming that java.security.MessageDigest implements the required digests, I noticed we didn't have a way to easily get at the underlying Java byte[] array in (SIMPLE-VECTOR (UNSIGNED-BYTE 8)). So, I patched the underlying Java code as attached, and wrote the following routine:
(defun sha-256 (path) (let ((buffer (make-array 8192 :element-type '(unsigned-byte 8)) (digest (jstatic "getInstance" "java.security.MessageDigest" "SHA-256"))) (with-open-file (in path :element-type '(unsigned-byte 8)) (loop :for bytes = (read-sequence buffer in) :while (plusp bytes) :do (jcall-raw "update" digest (make-immediate-object buffer) 0 bytes)) (jcall "digest" digest)))))
But then I noticed that MAKE-IMMEDIATE-OBJECT calls JOBJECT-LISP-VALUE which makes no conceptual sense. The doc for MAKE-IMMEDIATE-OBJECT reads "[a]ttempts to coerce a given Lisp object into a java-object" while the doc JOBJECT-LISP-VALUE reads "Attempts to coerce JAVA-OBJECT into a Lisp object". This seems wrong here, but I don't understand how exactly. I think we need to distinguish three types of values here: JAVA-OBJECT, LispObject, and the "raw" value of the Java. How should we really structure our Java FFI here?
With my patch, for an underlying lisp value of type (SIMPLE-VECTOR (UNSIGNED-BYTE 8)), both MAKE-IMMEDIATE-OBJECT and JOBJECT-LISP-OBJECT return the byte[] array. I would think that MAKE-IMMEDIATE-OBJECT should just do this, which JOBJECT-LISP-OBJECT should .... return an error if its argument isn't of type JAVA-OBJECT? I don't know. I am confused.
Can someone help me understand the right way forward here?
-- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
armedbear-devel mailing list armedbear-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
On 6/11/11 09:06 , Alan Ruttenberg wrote:
I've always disliked make-immediate-object. Really it serves the purpose of disambiguating a few overloaded values in lisp. I'd like to deprecate it in favor of specific constants for java-true, java-false and null-pointer (the latter is already available as function).
Unless anyone speaks up against this, I will make this change to the Java API.
It seems dangerous to assume that one can get some java array as the underlying object below a lisp array, what with all the hair that lisp arrays can have. Better to allocate your buffer with (jnew-array "byte" 8192) and not get into this game. Or have a documented policy of how java objects correspond to lisp objects and promise to support it forever.
I agree that we probably shouldn't go down the path of exposing the "internal" byte array here.
But I still couldn't find a way to covert a Lisp array of en masse to a Java byte[] structure.
One can't construct a byte from a Lisp integer with JAVA:JNEW
CL-USER> (jnew "byte" 0)
fails with #<JAVA-EXCEPTION java.lang.NoSuchMethodException: byte(java.lang.Integer) {39397F0A}>.
Same for
CL-USER> (jnew "java.lang.Byte" 0)
for the same reason.
One can use the Byte(string) constructor
CL-USER> (jnew "java.lang.Byte" "0")
but that seems kludgy in the extreme. Does anybody have a way to create a JAVA-OBJECT containing a byte? Did I miss something basic?
And JNEW-ARRAY-FROM-ARRAY on a Lisp vectors of numbers also fails, so there really seems to be another lacunae in our API.
In order to work through this, I modified ABCL in [r13327]() and [r13328]() to get
CL-USER> (jcoerce 0 "byte")
to return a "boxed" byte.
and added the ability to form a byte[] from the appropriate Lisp vector.
CL-USER> (type-of buffer) (SIMPLE-ARRAY (UNSIGNED-BYTE 8) (8192)) CL-USER> (jnew-array-from-array "byte" buffer) #<jarray [B@61d60df3 {2C2DFEB3}>
[r13327]: http://trac.common-lisp.net/armedbear/changeset/13327 [r13328]: http://trac.common-lisp.net/armedbear/changeset/13328
The conversion maps any number to its signed twos-complement value modulo 256, which I think is the right thing.
Please pipe up if this is the wrong way to move the ABCL Java API forward to more usability.
On 14-Jun-2011 10:49 PM, "Mark Evenson" evenson@panix.com wrote:
On 6/11/11 09:06 , Alan Ruttenberg wrote:
I've always disliked make-immediate-object. Really it serves the purpose of disambiguating a few overloaded values in lisp. I'd like to deprecate it in favor of specific constants for java-true, java-false and null-pointer (the latter is already available as function).
Unless anyone speaks up against this, I will make this change to the Java API.
It seems dangerous to assume that one can get some java array as the underlying object below a lisp array, what with all the hair that lisp arrays can have. Better to allocate your buffer with (jnew-array "byte" 8192) and not get into this game. Or have a documented policy of how java objects correspond to lisp objects and promise to support it forever.
I agree that we probably shouldn't go down the path of exposing the "internal" byte array here.
But I still couldn't find a way to covert a Lisp array of en masse to a Java byte[] structure.
One can't construct a byte from a Lisp integer with JAVA:JNEW
CL-USER> (jnew "byte" 0)
fails with #<JAVA-EXCEPTION java.lang.NoSuchMethodException: byte(java.lang.Integer) {39397F0A}>.
Same for
CL-USER> (jnew "java.lang.Byte" 0)
for the same reason.
One can use the Byte(string) constructor
CL-USER> (jnew "java.lang.Byte" "0")
but that seems kludgy in the extreme. Does anybody have a way to create a JAVA-OBJECT containing a byte? Did I miss something basic?
Yeah, I've forgotten and "worked around" this a few times. I think I have always converged on (jcall-raw "byteValue" 23) for getting single byte values.
And JNEW-ARRAY-FROM-ARRAY on a Lisp vectors of
I have been filling in the array content element by element, after getting individual byte values as above.
Yong
On 6/14/11 12:49 , Mark Evenson wrote:
On 6/11/11 09:06 , Alan Ruttenberg wrote:
I've always disliked make-immediate-object. Really it serves the purpose of disambiguating a few overloaded values in lisp. I'd like to deprecate it in favor of specific constants for java-true, java-false and null-pointer (the latter is already available as function).
Unless anyone speaks up against this, I will make this change to the Java API.
After introducing the constants +NULL+, +TRUE+, and +FALSE+ for the corresponding primitives types wrapped in JAVA_OBJECT, I have as threatened [marked MAKE-IMMEDIATE-OBJECT as deprecated][r13359].
[r13359] :http://trac.common-lisp.net/armedbear/changeset/13359
armedbear-devel@common-lisp.net
-
Alan Ruttenberg -
Mark Evenson -
Theam Yong Chew