On Thu, Jan 7, 2010 at 4:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:

On Thu, Jan 7, 2010 at 6:39 AM, Alan Ruttenberg alanruttenberg@gmail.com wrote:

...

I'm trying to use the new jcall code in place of jss calls. However I am getting an error I don't understand.

In the following:

(defun directory-in-jar (pathname)  (let* ((jarfile (subseq (namestring (pathname-device pathname)) 9))         (rest-pathname (namestring (make-pathname :directory (pathname-directory pathname)                                                   :name (pathname-name pathname)                                                   :type (pathname-type pathname)))))    (if (or (position #* (namestring rest-pathname))            (wild-pathname-p rest-pathname))        (let ((jar (jnew  "java.util.zip.ZipFile" jarfile)))          (let ((els (jcall "entries" jar)))            (loop while (#"hasMoreElements" els)               for name = (jcall "getName" (#"nextElement" els))               when (pathname-match-p (concatenate 'string "/" name) rest-pathname)               collect (make-pathname :device (pathname-device pathname)                                      :name (pathname-name name)                                      :type (pathname-type name)                                      :directory `(:absolute ,@(cdr (pathname-directory name)))))))        (let ((truename (probe-file-in-jar pathname)))          (if truename              (list truename)              nil)))))

If I replace the call (#"hasMoreElements" els) with (jcall "hasMoreElements" els) I get the error:

java.lang.IllegalAccessException: Class org.armedbear.lisp.Java can not access a member of class java.util.zip.ZipFile$3 with modifiers "public"        at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)        at java.lang.reflect.Method.invoke(Method.java:583)        at org.armedbear.lisp.Java.jcall(Java.java:660) ...

If I replace the call  (#"nextElement" els) with  (jcall "nextElement" els) I get the error

java.lang.IllegalAccessException: Class org.armedbear.lisp.Java can not access a member of class java.util.zip.ZipFile$3 with modifiers "public volatile"        at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)        at java.lang.reflect.Method.invoke(Method.java:583)        at org.armedbear.lisp.Java.jcall(Java.java:660)

This from http://svn.mumble.net:8080/svn/lsw/trunk/patches/jarfile.lisp where I'm starting to add more path functions that work with the jar:file: pathname scheme. Feel free to steal and incorporate into abcl.

This code fails too with the same error:

(let ((jar (jnew  "java.util.zip.ZipFile" "/home/alessio/abcl/pippo.jar")))   (let ((els (jcall "entries" jar)))     (let ((method (jmethod (jclass-of els) "hasMoreElements")))       (jcall method els))))

as does its translation to Java:

ZipFile jar = new ZipFile("/home/alessio/abcl/pippo.jar"); Object els = jar.entries(); Method method = els.getClass().getMethod("hasMoreElements"); method.invoke(els);

so it's some quirk of Java reflection that I'm not aware of. In fact, calling method.setAccessible(true); before invoking it removes the error. Maybe the inner classes ZipFile$x are not public, and that is sufficient to cause an IllegalAccessException? In any case, ABCL's jcall currently makes no attempts at ensuring the method is accessible before calling it.

hmm, it's weird. In the Java code I posted, using Method method = java.util.Enumeration.class.getMethod("hasMoreElements"); works, even without setAccessible(true)! Although with dynamic dispatch the method that gets called is really the same...

And it's even weirder: iterating through els.getClass().getMethods() gives

... public java.util.zip.ZipEntry java.util.zip.ZipFile$2.nextElement() throws java.util.NoSuchElementException declared in class java.util.zip.ZipFile$2 public java.lang.Object java.util.zip.ZipFile$2.nextElement() declared in class java.util.zip.ZipFile$2 ...

clearly one of the two is inherited from Enumeration, but it gets reported as being defined in the inner class, resulting in an apparently duplicated method!

There are several bugs filed for the JVM about this or similar issues, dating as far back as Java 1.2 (!!!); examples are http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4819108, http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4071957. To my understanding, implementing the proper checks efficiently in order to solve the problem would involve changing the class file format, so this bug has been forever postponed.

At this point, we can either

1. ignore the problem, 2. always call setAccessible(true) before calling a method, 3. try to find the overridden method as up as possible in the class hierarchy before calling it, hoping to get at some point to a public class/interface

2. and 3. would make all method calls through jcall less efficient; however, we could enable one of those techniques only when some additional parameter is passed to jcall (e.g. the try-harder that was proposed some time ago for jfield).

Bye, Ale

On Thu, Jan 7, 2010 at 6:48 PM, Alan Ruttenberg alanruttenberg@gmail.com wrote:

On Thu, Jan 7, 2010 at 11:37 AM, Alessio Stalla alessiostalla@gmail.com wrote:

...

On Thu, Jan 7, 2010 at 4:32 PM, Alessio Stalla alessiostalla@gmail.com wrote:

...

On Thu, Jan 7, 2010 at 6:39 AM, Alan Ruttenberg alanruttenberg@gmail.com wrote:

At this point, we can either

1. ignore the problem,
2. always call setAccessible(true) before calling a method,
3. try to find the overridden method as up as possible in the class

hierarchy before calling it, hoping to get at some point to a public class/interface

2. and 3. would make all method calls through jcall less efficient;

however, we could enable one of those techniques only when some additional parameter is passed to jcall (e.g. the try-harder that was proposed some time ago for jfield).

Bye, Ale

I believe that the code that jss uses a method cache and therefore only needs to do the setAccessible call once. Have you looked at what jsint.Invoke does or considered simply adopting it for ABCL?

http://jscheme.sourceforge.net/jscheme/doc/api/jsint/Invoke.html

We designed the following solution:

every Java object (instance of JavaObject) carries a reference to its "intended class", which must be a supertype of the object's class. The intended class is used to access the object's methods via reflection; only if a method is not found in the intended class the more specific object class is used, and only if it's public.

How is the intended class determined? There are two possibilities:

- automatically, by looking at the method return type (for objects returned by a method call), field type (for objects read from a field), or class of the object (when it is constructed explicitly with jnew) - manually, through a new operator - (jcoerce obj class)

The automatic way covers the common case of methods returning an instance of a class or interface implemented via a non-public inner class, which was the source of the bug.

For more complex cases - for example when an object is an instance like the above but implements several different interfaces, and must be accessed specifically through one of them, which might not be the one that has been chosen automatically - jcoerce comes to help.

This patch is now applied on trunk (rev. 12345) and you are encouraged to try it and comment on it.

I haven't benchmarked it, but I don't expect a performance degradation, just a slightly increased memory usage due to the extra reference to the intended class stored in every JavaObject.

Bye, Alessio

...

This patch is now applied on trunk (rev. 12345) and you are encouraged to try it and comment on it.

I haven't benchmarked it, but I don't expect a performance degradation, just a slightly increased memory usage due to the extra reference to the intended class stored in every JavaObject.

Bye, Alessio

There's a regression:

Error loading /Users/alanr/repos/lsw/trunk/jss/armedbear-0.18.0-dev-darwin-unknown/invoke.abcl at line 56 (offset 3507) Debugger invoked on condition of type TYPE-ERROR: java.lang.Boolean is not assignable to boolean Restarts: 0: RETRY Retry performing #<ASDF:LOAD-OP (:VERBOSE T) {26EEE5}> on #<ASDF:CL-SOURCE-FILE "invoke" {ABE92B}>. 1: ACCEPT Continue, treating #<ASDF:LOAD-OP (:VERBOSE T) {26EEE5}> on #<ASDF:CL-SOURCE-FILE "invoke" {ABE92B}> as having been successful. [1] CL-USER(1): :bt 0: (SYSTEM:BACKTRACE) 1: (INVOKE-DEBUGGER #<TYPE-ERROR {8B356D}>) 2: org.armedbear.lisp.Lisp.error(Lisp.java:353) 3: org.armedbear.lisp.JavaObject.javaInstance(JavaObject.java:238) 4: org.armedbear.lisp.Java.jstatic(Java.java:389) 5: org.armedbear.lisp.Java.access$600(Java.java:50) 6: org.armedbear.lisp.Java$9.execute(Java.java:434) 7: org.armedbear.lisp.Primitive.execute(Primitive.java:174) [1] CL-USER(2):

Kind of curious: Why are you not just using the jsint implementation?

Well, one reason being that their implemetation requires a call to setAccessible() which is defined to be allowed to raise a security exception. The method Alessio implemented (and I figured out long ago in order to implement USOCKET on ABCL) doesn't have that drawback. When this strategy can't work, well, I guess we'll need to revisit and go the security exception route.

It's been in use for quite a while and it would certainly make my life easier to be able to continue to use that interface. For example, I use their findMethod implementation to get a method so I can optimize certain loops by avoiding dynamic lookup each time.

Right. We want to start doing caching like that too, based on the object class, method name and the argument classes. That would be what you need, right?

Bye,

Erik.

On Fri, Jan 8, 2010 at 4:21 PM, Erik Huelsmann ehuels@gmail.com wrote:

...

Kind of curious: Why are you not just using the jsint implementation?

Well, one reason being that their implemetation requires a call to setAccessible() which is defined to be allowed to raise a security exception. The method Alessio implemented (and I figured out long ago in order to implement USOCKET on ABCL) doesn't have that drawback. When this strategy can't work, well, I guess we'll need to revisit and go the security exception route.

If it happens to raise a security exception, one can deal with it then. I've used the jsint implementation for a long time now and I don't recall that happening once.

...

It's been in use for quite a while and it would certainly make my life easier to be able to continue to use that interface. For example, I use their findMethod implementation to get a method so I can optimize certain loops by avoiding dynamic lookup each time.

Right. We want to start doing caching like that too, based on the object class, method name and the argument classes. That would be what you need, right?

Yes. However I am concerned that this is work that need not be done, freeing up time to do other things. The jsint implementation does what is necessary and is the product of a good deal of thought and experience. Seems to me that using mature code is preferable to re-implementing stuff.

Regards, Alan

(and thanks for your efforts, in any case!)

On Fri, Jan 8, 2010 at 5:39 PM, Alessio Stalla alessiostalla@gmail.com wrote:

My main objection against jsint is that it requires an additional jar on the classpath, of which it uses just a few classes. But read on.

Just to reply to this point.

I wouldn't distribute it that way. I'd either extract just the class files necessary and have the build include them in abcl.jar, or just get the relevant source files, put them in the ABCL source tree, and adapt them as necessary.

Certainly now that we have the ability to load asds directly from a jar my plan is to distribute my code as a single jar in which I've included abcl, the contents of all the jars that my tool depends on, and all the compiled lisp code. Goodbye class path.

Any yes, of course I can continue to use jsint myself, however as I found it quite useful I was hoping to spread the goodness around more widely :)

Cheers, Alan