I hope this is not a stupid question, but: Why does jcall use reflection?
Doesn't ABCL compile lisp to byte code? If one could statically declare to the compiler the intended Java class, couldn't method invocations be compiled instead of being performed via reflection?
OTOH, I don't know how ABCL handles interpretation versus compilation, so that might be an issue.
On Sat, Jan 9, 2010 at 12:28 AM, Axel Rauschmayer axel@rauschma.de wrote:
I hope this is not a stupid question, but: Why does jcall use reflection?
Doesn't ABCL compile lisp to byte code? If one could statically declare to the compiler the intended Java class, couldn't method invocations be compiled instead of being performed via reflection?
OTOH, I don't know how ABCL handles interpretation versus compilation, so that might be an issue.
I had a talk today with Erik about this topic, so I suppose I'm informed enough to speak ;)
To make it short: reflection could be eliminated in some cases, but it's quite costly in terms of development effort.
The long explanation: there are several reasons. Keep in mind that ABCL has both an interpreter and a compiler.
- In interpreted code, reflection is the only possibility. - In compiled code: - some method invocations through jcall cannot be resolved at compile time - extreme example: (jcall (read) nil). - others could be resolved. How many depends on the effort we can put into the compiler: it could recognize only calls to a method which literally appears in source code - (jcall (jmethod "foo" "bar") baz) - or, by doing type inference, more cases could be discovered. The compiler currently does nothing in that regard, except caching the method metaobjects per call-site when possible. So in the foo-bar-baz example above, method "bar" of class "foo" is only resolved once, at load time, and then reused - for that particular call site.
It is dubious whether enhancing the compiler to eliminate reflection for some jcall invocations would be worth the effort. Since Java objects are boxed in ABCL, the compiler would need to generate code to unbox the arguments and box the return value, and place appropriate exception handlers. This would likely increase the size and complexity of the generated code quite a lot. Also, I believe that to optimize a significant number of jcall invocations we need type inference, and that's not an easy task. Of course, the implementation of jcall has to do the very same boxing/unboxing, besides the reflective call itself; it is inevitably less efficient. But I think that, unless your code is made mostly of calls to Java methods, the difference is tolerable.
hth, Alessio
It is dubious whether enhancing the compiler to eliminate reflection for some jcall invocations would be worth the effort. Since Java objects are boxed in ABCL, the compiler would need to generate code to unbox the arguments and box the return value, and place appropriate exception handlers. This would likely increase the size and complexity of the generated code quite a lot. Also, I believe that to optimize a significant number of jcall invocations we need type inference, and that's not an easy task. Of course, the implementation of jcall has to do the very same boxing/unboxing, besides the reflective call itself; it is inevitably less efficient. But I think that, unless your code is made mostly of calls to Java methods, the difference is tolerable.
Naively (=not knowing how it all fits into the bigger ABCL picture), I would expect Java objects not to be boxed and the compiler to be helped by some kind of foreign function declaration. That is, all types are declared statically, dynamic single dispatch should be handled by the JVM. This would mean an initial point of pain (writing the declarations), but elegant usage afterwards. A small script should be able to generate this kind of declarations.
(defjvar foo java.util.StringBuilder) (defjfun sb-to-string java.util.StringBuilder)
(setf foo (new java.util.StringBuilder)) (sb-to-string foo)
Am I making sense or is this off the mark?
The good thing is that any kind of Java calls are always going to be encapsulated inside a lisp interface, so that migrating to something different later is never an issue. I have a feeling that supporting Java's collection APIs (including iterators and Iterable) and Java's streams would be helpful when connecting from ABCL, but I'll have to work some more on my ABCL API for Sesame [1], to be sure.
On Sat, Jan 9, 2010 at 2:17 AM, Axel Rauschmayer axel@rauschma.de wrote:
It is dubious whether enhancing the compiler to eliminate reflection for some jcall invocations would be worth the effort. Since Java objects are boxed in ABCL, the compiler would need to generate code to unbox the arguments and box the return value, and place appropriate exception handlers. This would likely increase the size and complexity of the generated code quite a lot. Also, I believe that to optimize a significant number of jcall invocations we need type inference, and that's not an easy task. Of course, the implementation of jcall has to do the very same boxing/unboxing, besides the reflective call itself; it is inevitably less efficient. But I think that, unless your code is made mostly of calls to Java methods, the difference is tolerable.
Naively (=not knowing how it all fits into the bigger ABCL picture), I would expect Java objects not to be boxed and the compiler to be helped by some kind of foreign function declaration. That is, all types are declared statically, dynamic single dispatch should be handled by the JVM. This would mean an initial point of pain (writing the declarations), but elegant usage afterwards. A small script should be able to generate this kind of declarations.
(defjvar foo java.util.StringBuilder) (defjfun sb-to-string java.util.StringBuilder)
(setf foo (new java.util.StringBuilder)) (sb-to-string foo)
Am I making sense or is this off the mark?
The main reason for boxing Java objects is probably to ensure that ABCL can deal with all objects through a common interface - the base class LispObject. Had not this been the case, the methods in most classes in the Java part of ABCL would have had to accept arbitrary Objects and do type checks at runtime:
LispObject type; if(parameter instanceof LispObject) { type = ((LispObject) parameter).typeOf(); } else { type = Symbol.JAVA_OBJECT; }
which would have been ugly and inefficient.
That said, it would be possible for the compiler to avoid some of the boxing/unboxing in certain cases, perhaps with the help of declarations.
The good thing is that any kind of Java calls are always going to be encapsulated inside a lisp interface, so that migrating to something different later is never an issue. I have a feeling that supporting Java's collection APIs (including iterators and Iterable) and Java's streams would be helpful when connecting from ABCL, but I'll have to work some more on my ABCL API for Sesame [1], to be sure.
wrt. streams, the machinery is pretty much already there - ABCL stream classes are wrappers around Java streams and reader/writers. There's currently no way to access that from Lisp except manually instantianting ABCL's stream classes with the Java FFI. One possibility is to add the capability to automatically translate streams returned from Java, like it already happens with strings and numbers; I have code to do that somewhere, an abandoned experiment. My only concern is efficiency: streams are going to be translated (boxed) even if not used as Lisp streams, and unboxed when passed again to Java, and this operation is heavier than usual because ABCL's streams are not mere boxes, they have some initialization logic. An alternative would be to provide operators to do the conversion manually, e.g. (as-lisp-stream java-stream). Then the burden would be on the programmer, but the translation would only be performed when needed.
Bye, Alessio
2010/1/9 Axel Rauschmayer axel@rauschma.de:
I hope this is not a stupid question, but: Why does jcall use reflection?
For various historical reasons and for certain complexity reasons, I guess.
Doesn't ABCL compile lisp to byte code? If one could statically declare to the compiler the intended Java class, couldn't method invocations be compiled instead of being performed via reflection?
The question is how we handle cases where people lie the static type. :) Java compiler makes all kinds of static checks that we might not want to reimplement, and the java compiler api is available only since java 1.6. Having that compiler api would allow us to convert certain kind of imaginary static-java-call to java code and compile that, without reinventing wheels. We still target 1.5 and avoid forcing people to use 1.6. Then again, we could just as well be optimistic and generate proper exception handling code for cases where people misuse the static declation. It's a bit of work to get right, I suppose, so we would need someone to do the hard work.
OTOH, I don't know how ABCL handles interpretation versus compilation, so that might be an issue.
Interpreter could just use reflection, whereas compiler could try and use direct calls, with some safeguarding as mentioned above. I don't think it's at all impossible to provide a non-reflective way to invoke java functions, but there's a bit of work that it requires and so far it just hasn't been done, I guess.
Other people may have insights for this, Erik has been toying with the idea of compiling some java calls to non-reflective calls as far as I remember. Could be that I'm missing some impediments, of course.
armedbear-devel@common-lisp.net
-
Alessio Stalla -
Axel Rauschmayer -
Ville Voutilainen