Hi Blake, I'm the original author of JSS and I completely agree. Have a look at JSS and see how it feels. Aside from the syntax, note that imports are not necessary, ABCL has incorporated dynamic additions to the classpath, one cnanan refer to classes using symbols or strings that only need to match uniquely to the end of the fully qualified class name and there are extensions to asdf that let you have jar files specified as part of system definitions. The only thing that has stumped me is how to dynamically add native libraries - it would be great if you or someone Else could figure out a way to let that happen. Also I think there's one remaining bug with method look up that we've reported that I don't think is addressed yet. I suspect it's the case that modifying a ABCL to use symbols as the function names is feasible just by modifying the readtable so you might give that a try. In the end I decided that because of the case sensitivity and the overlap with existing function names it was clearer to have a distinct syntax. Here's another item that I think would be a very nice addition towards the goal we both agree is worthwhile. I find myself regularly finding some code snippet or test program written in java that I want to use in abcl. Wouldn't it be sweet if there was a slime addition paste-as-jss that did the transforms necessary to translate from java syntax to jss in ABCL. This wouldn't have to be a full java parser - even support for a subset of statements and the ability to ignore much of the declaration nonsense that jss/abcl doesn't need, and the ability to leave alone what it doesn't know how to translate, would be a real help for lisp programmers who want to use java. Best, Alan On Saturday, February 16, 2013, Blake McBride wrote:

I'd like to add that there are many high quality and free lisp systems available. The thing that makes ABCL uniquely interesting is its close association to Java. It gives ABCL a much better ability to leverage off of the existing technology (libraries) built in Java. While focusing on ABCL's reliability and conformance to standards is very important, I wouldn't loose sight of where the attraction really is. If the main attraction to ABCL for lisp programmers is its tight integration to Java, than the degree of ABCL's appeal will be directly related to the ease and power of its integration to Java.

Just one opinion...

Blake McBride

On Sat, Feb 16, 2013 at 7:54 AM, Alessio Stalla <alessiostalla@gmail.com>wrote:

...

...

I have an idea for a largely seamless integration method for ABCL with Java that involves changing the ABCL REPL. It would work as follows.

1. From within Java or lisp, have the ability to register any number of Java classes with ABCL.

2. When executing lisp code of the form (fun args ...), try running the code as regular lisp code as it does now, but if it fails to find the

On Sat, Feb 16, 2013 at 2:32 PM, Blake McBride <blake@arahant.com> wrote: lisp

...

function and has at least one argument then:

3. Examine the arguments and, through reflection, try to find a Java method (with the same name as the lisp function) in one of the registered Java classes that matches the number and types of arguments and execute that. So, it would auto translate (in effect):

(fun arg1 arg2 arg3 ...)

into Java

arg1.fun(arg2, arg3, ...);

4. There must be better auto-translation of lisp <-> java data types to include arrays and lists in order for this to work. So, in other words, when executing a lisp function passing a lisp array as one of the arguments, the Java reflection query will see a Java array argument type. It will also be auto-converted when making the call.

5. There must be some sort of cacheing mechanism to avoid redundant Java reflection calls.

6. This mechanism must take into account and correctly handle similar Java method names with different number and types of arguments both within and across different classes. It must also handle the fact that methods may be declared in Java superclasses.

7. This functionality can be enabled or disabled through some (Java and/or lisp function). Off by default to avoid troubles.

Adding this functionality would make ABCL a plugin-and-go extension to Java. Any code could be written in Java or lisp immediately without complicated and cumbersome interface/translation code in most cases. There would be no huge initial hill to climb (to create all the interface code), and the usage would be natural and uncomplicated.

Thanks.

I never really used it, but it seems to me that you're mostly describing JSS. It is not so seamlessly integrated with the REPL, but it has reader macros that allow you to write (#"someMethod" this arg1 arg2 ...). ABCL (and thus JSS) knows how to find the best matching method based on argument types, taking subclassing into account.

Lisp lists are not automatically translated to Java lists because they are made of conses, but a cons does not necessarily translate to a list, it might represent an improper list, a tree, a graph... on the other hand, you can use Java lists natively as Lisp sequences in ABCL, so you might be better off using Java lists from the start, if you find yourself converting lists back and forth frequently.

Alessio

On Sun, Feb 17, 2013 at 11:59 AM, Mark Evenson <evenson@panix.com> wrote:

On 2/17/13 1:53 AM, Alan Ruttenberg wrote: […]

...

The only thing that has stumped me is how to dynamically add native libraries - it would be great if you or someone Else could figure out a way to let that happen. Also I think there's one remaining bug with method look up that we've reported that I don't think is addressed yet.

Depending on what you mean by dynamic, [CFFI][] uses JNA, possibly loading it from the network via ABCL-ASDF, to interface with native code. The version git is a little more advanced than the one currently distributed with Quicklisp. Most of the the outstanding problems involving correctly converting Lisp values when utilizing callbacks. What else would you like to have here?

I mean code that is loaded by System.loadLibrary, in which the name of the library is given, but it is looked up on some path. It is documented that you could set java.library.path at launch time, but I don't see any way to modify it at run time. I want an analogous situation as with dynamic classpath. Sorry I wasn't clear.

[CFFI]:http://common-lisp.net/project/cffi/

[…]

--

"A screaming comes across the sky. It has happened before, but there is nothing to compare it to now."

_______________________________________________ armedbear-devel mailing list armedbear-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel

It was 229 I was referring to. I see it is marked as fixed (yeah!). Sorry for the red herring. Thanks, Mark! -Alan On Sun, Feb 17, 2013 at 12:08 PM, Mark Evenson <evenson@panix.com> wrote:

On 2/17/13 1:53 AM, Alan Ruttenberg wrote:

...

[…] Also I think there's one remaining bug with method look up that we've reported that I don't think is addressed yet.

[…]

I think you are referring to [#259][] for which the status is that your original form

(jss:jarray-to-list (java:jnew-array (jclass "int") 40))

has been working for some time.

[#259]: http://trac.common-lisp.net/armedbear/ticket/259

The equivalent form not utilizing JSS, namely

(jstatic "asList" "java.util.Arrays" (java:jnew-array (jclass "int") 1))

still fails. The problems stem from how Java-the-language erases parametrized type information at runtime, and how the JAVA package currently tries to match arguments and how it coerces arrays of primitives (it currently doesn't implicitly).

As a workaround, don't use arrays of primitive types, instead using the "boxed" Java type:

(jstatic "asList" "java.util.Arrays" (java:jnew-array (jclass "java.lang.Integer") 1))

There should be negligible speed differentials on contemporary JVM implementations from what I understand, and if you were that worried about memory, you wouldn't be using Java in the first place, now would you?

--

"A screaming comes across the sky. It has happened before, but there is nothing to compare it to now."

I thought of one more thing that had been bouncing around. Compiler support for speed optimization. Currently I use the with-constant-signature macro, which lets me express that within a certain block a specified method will always be called with the same signature. The implementation is to cache the method looked up by the first call, and call that method throughout the rest of the block. It's pretty helpful for bumming speed on tight looks, such as when iterating through a large array or list. By doing this you save the lookup against the cache (and it makes a difference in some code). Ideally this would be expressed as a compiler directive like (declare (constant-signature #"foo")) , which, if speed optimization was high enough, would do as I say above. I suppose the same declaration/optimization could be used for Clos method calls. Finally, I haven't looked recently looked at the syntax for defining java classes and methods in lisp. jinterface-implementation gets the job done (though i used to have to be careful about exceptions - don't know if I still need to). But one might think a syntax closer to clos would be more familiar. -Alan On Sat, Feb 16, 2013 at 7:53 PM, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:

Hi Blake,

I'm the original author of JSS and I completely agree. Have a look at JSS and see how it feels. Aside from the syntax, note that imports are not necessary, ABCL has incorporated dynamic additions to the classpath, one cnanan refer to classes using symbols or strings that only need to match uniquely to the end of the fully qualified class name and there are extensions to asdf that let you have jar files specified as part of system definitions.

The only thing that has stumped me is how to dynamically add native libraries - it would be great if you or someone Else could figure out a way to let that happen. Also I think there's one remaining bug with method look up that we've reported that I don't think is addressed yet.

I suspect it's the case that modifying a ABCL to use symbols as the function names is feasible just by modifying the readtable so you might give that a try. In the end I decided that because of the case sensitivity and the overlap with existing function names it was clearer to have a distinct syntax.

Here's another item that I think would be a very nice addition towards the goal we both agree is worthwhile. I find myself regularly finding some code snippet or test program written in java that I want to use in abcl. Wouldn't it be sweet if there was a slime addition paste-as-jss that did the transforms necessary to translate from java syntax to jss in ABCL. This wouldn't have to be a full java parser - even support for a subset of statements and the ability to ignore much of the declaration nonsense that jss/abcl doesn't need, and the ability to leave alone what it doesn't know how to translate, would be a real help for lisp programmers who want to use java.

Best, Alan

On Saturday, February 16, 2013, Blake McBride wrote:

...

I'd like to add that there are many high quality and free lisp systems available. The thing that makes ABCL uniquely interesting is its close association to Java. It gives ABCL a much better ability to leverage off of the existing technology (libraries) built in Java. While focusing on ABCL's reliability and conformance to standards is very important, I wouldn't loose sight of where the attraction really is. If the main attraction to ABCL for lisp programmers is its tight integration to Java, than the degree of ABCL's appeal will be directly related to the ease and power of its integration to Java.

Just one opinion...

Blake McBride

On Sat, Feb 16, 2013 at 7:54 AM, Alessio Stalla <alessiostalla@gmail.com> wrote:

...

On Sat, Feb 16, 2013 at 2:32 PM, Blake McBride <blake@arahant.com> wrote:

...

I have an idea for a largely seamless integration method for ABCL with Java that involves changing the ABCL REPL. It would work as follows.

1. From within Java or lisp, have the ability to register any number of Java classes with ABCL.

2. When executing lisp code of the form (fun args ...), try running the code as regular lisp code as it does now, but if it fails to find the lisp function and has at least one argument then:

3. Examine the arguments and, through reflection, try to find a Java method (with the same name as the lisp function) in one of the registered Java classes that matches the number and types of arguments and execute that. So, it would auto translate (in effect):

(fun arg1 arg2 arg3 ...)

into Java

arg1.fun(arg2, arg3, ...);

4. There must be better auto-translation of lisp <-> java data types to include arrays and lists in order for this to work. So, in other words, when executing a lisp function passing a lisp array as one of the arguments, the Java reflection query will see a Java array argument type. It will also be auto-converted when making the call.

5. There must be some sort of cacheing mechanism to avoid redundant Java reflection calls.

6. This mechanism must take into account and correctly handle similar Java method names with different number and types of arguments both within and across different classes. It must also handle the fact that methods may be declared in Java superclasses.

7. This functionality can be enabled or disabled through some (Java and/or lisp function). Off by default to avoid troubles.

Adding this functionality would make ABCL a plugin-and-go extension to Java. Any code could be written in Java or lisp immediately without complicated and cumbersome interface/translation code in most cases. There would be no huge initial hill to climb (to create all the interface code), and the usage would be natural and uncomplicated.

Thanks.

I never really used it, but it seems to me that you're mostly describing JSS. It is not so seamlessly integrated with the REPL, but it has reader macros that allow you to write (#"someMethod" this arg1 arg2 ...). ABCL (and thus JSS) knows how to find the best matching method based on argument types, taking subclassing into account.

Lisp lists are not automatically translated to Java lists because they are made of conses, but a cons does not necessarily translate to a list, it might represent an improper list, a tree, a graph... on the other hand, you can use Java lists natively as Lisp sequences in ABCL, so you might be better off using Java lists from the start, if you find yourself converting lists back and forth frequently.

Alessio

Lest there be any confusion, I deeply appreciate the ABCL system and all those who contribute to it. Blake McBride On Tue, Feb 19, 2013 at 4:18 AM, Mark Evenson <evenson@panix.com> wrote:

On Feb 16, 2013, at 1622 , Blake McBride <blake@arahant.com> wrote:

...

I'd like to add that there are many high quality and free lisp systems available. The thing that makes ABCL uniquely interesting is its close association to Java. It gives ABCL a much better ability to leverage off of the existing technology (libraries) built in Java. While focusing on ABCL's reliability and conformance to standards is very important, I wouldn't loose sight of where the attraction really is. If the main attraction to ABCL for lisp programmers is its tight integration to Java, than the degree of ABCL's appeal will be directly related to the ease and power of its integration to Java.

Just to clarify the aims of Armed Bear Common Lisp from my standpoint:

We aim to make ABCL a conforming, performant ANSI Common Lisp implementation as the platform upon we experiment, developing easier Java integration as we iterate through usage and feedback. This goal is currently (mostly[^1]) reflected in how ABCL is packaged: abcl.jar contains the core conforming ANSI implementation, whereas abcl-contrib.jar contains the goodies which make working with Java quite a bit easier. With features that are specific to the hosting JVM, such as allowing CL:PATHNAME to refer to URIs or treating java.util.List descendants as a user extensible sequence, we have striven to use existing extension methods as much as possible to follow the "principle of least surprise".

As such, we believe we offer the best of both worlds. As a first-class Common Lisp implementation which runs on the JVM, ABCL benefits from the contemporary Lisp ecosystem of cross-implementation libraries exemplified as those distributed via Quicklisp. In addition to contemporary advances, by stressing core conformance, we allow applications and research developed decades ago, such as MACSYMA, to be able to be run in new environments, for new purposes.

As our community contributes new ideas to making working with Java easier such as JSS and JFLI, we will attempt to include them in ABCL-CONTRIB.

And, as always, implementation is the sincerest form of flattery.

[^1]: "mostly" because the system obviously needs to have some hooks into Java as its implementation language, which form the basis of the symbols in the JAVA package, and to some extent SYSTEM and EXTENSIONS. The user-extensible collections for java sequences in JAVA-COLLECTIONS are sort of a historical exception to this "rule" as well.

-- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

_______________________________________________ armedbear-devel mailing list armedbear-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel

I only looked briefly at JSS but quickly formed a, possibly unfounded, opinion as follows The appearance of the calls looks special and awkward. In my case, nearly every line will be a call into my Java libraries. This would render the lisp code maximumly awkward, ugly, and difficult to read. The solution I had up to this point made the lisp code natural and native looking. I think the JSS code is nice if you have a few places you need to link into and didn't want to write the interface code. Using ABCL as an extension language to a large application is an entirely different story. In terms of returning lists and arrays, I have many places where, for example, a method would return a list of records. Using it as a lisp list when in lisp of very natural and works well with things like loop. Having to call Java to iterate through lists and arrays is, again, unnecessarily unnatural. I think using that extension to clos I proposed could solve my problem but would be complicated to get working right. I lean towards changing ABCL's REPL. It would work as it does now but if it can't find a function, it automatically tries finding a Java method before issuing a function not found error message. I think this would be pretty easy to implement and would automatically eliminate most interface code requirements, and render the interface natural to lisp. Thanks. Blake On Wed, Feb 20, 2013 at 5:08 AM, Mark Evenson <evenson@panix.com> wrote:

On Feb 19, 2013, at 1420 , Blake McBride <blake@mcbride.name> wrote:

...

Lest there be any confusion, I deeply appreciate the ABCL system and all those who contribute to it.

From my side, I didn't intend my message to seem defensive, more just a clarification of points that I have expressed to people, but not on the mailing list. And posts like yours are quite welcome.

I'm still pondering how to answer your proposals, but it is taking a long time to find the time.

Just two quick points:

1. Have you looked at JSS as Alessio suggests? Can you comment as to how it meets your numbered points?

2. Changing the call/return values from the Java FFI in changing java.lang.List to/from Lisp CONS as well as Java array to/from Lisp arrays would break existing code that expects otherwise. Any ideas on how to migrate?

-- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."