Thanks for the quick and detailed response.
I read that, and that did help a lot in understanding the new code. BTW, regarding the constructors, shouldn't you be able to define a method entry with name as :constructor and have that work? From looking at the function map-method-name in jvm-class-file.lisp, but not actually tested it yet.
What I wanted to achieve is to have methods that can be:
* arbitrary functions, including closures
* possibly redefined after the class has been loaded. Redefinition is
possible by using symbols as method implementations.
There's also a hidden goal that is to keep the implementation simple
enough, and compiling Lisp code to the body of methods in an arbitrary
class is not easy at the moment.
What explicitly is NOT a requirement is to have the generated class be
independent from ABCL (the methods are implemented in Lisp and will
always need runtime support). So, no matter how you arrange things,
you will never be able to distribute your class without abcl.jar. Note
that it is the same for Clojure, Scala, Groovy, etc.
I don't know specifically how Hadoop works, but, as I see things, you need to
1. provide abcl.jar + your class to it
2. have Hadoop execute enough code to load ABCL and initialize your class.
Right. This is the same in clojure where I implemented something similar, but the language was unfamiliar and the way to actually call compile felt clunky, but some of that surely was simply unfamiliarity.
So what I want to do is the following:
1. have abcl.jar and generated_filename_with_method_implementations.lisp in the classpath
2. Do the jvm equivalent of the following java code:
Interpreter interp = Interpreter.createInstance();
interp.eval("(load \"generated_filename_with_method_implementations.lisp\")");
in the static initialization block of the class.
Now, probably the key point is who loads your class.
* if you can arrange to have ABCL load it and pass it to Hadoop, you
should be good with the current design.
* if Hadoop wants to load it by itself from a jar, then yes, you have
to somehow dump the method implementations somewhere and have the
class auto-load them from there when it is initialized.
For the second case, I propose an intermediate design that is, imho,
simpler than dumping arbitrary forms: special-case the situation when
a method implementation is a symbol. In that case, directly generate
an instruction in the class file to find that symbol in its package
and assign it to the appropriate field. If the implementation of a
method is not a symbol nothing changes. That way, you will still need
to init ABCL before loading your class, but, when all your method
implementations are symbols, the class will be able to initialize
itself without external support. If some methods are proper functions,
instead, the class won't be self-contained, but will need some Lisp
code to be executed to be properly initialized.
Does it make sense to you? Do you find it acceptable? If yes, I can
give a shot at implementing it, if you want.
This certainly seems reasonable, but I don't understand the bit about "directly generate an instruction to find the package and do the assignement". Do I not need to change the compile infrastructure to get at the jvm representation of the (fdefinition name) ? The code in %jvm-compile (in compiler-pass2.lisp) seems to do the compilation to a (sys::%make-byte-array-output-stream) rather than to arbitrary output-streams. Don't know the reason for this. If this can be changed, we can do this for arbitrary forms, rather than just symbols.
ie, we can compile the relevant forms into classname_methodname.abcl and generate a static method call to Load.load(filename) in the static init block as before, to do the autoloading, and then we will know the LispObject name holding the jmethod and can assign that to the appropriate field as you suggest above.
Other than this (the compile issue), I find this entirely reasonable and would love it if you do implement it.
much regards,
Archi