[armedbear-devel] Performance of ABCL?
Hi! I've tested Maxima with ABCL. While it runs, which is rather cool, it's fairly slow! On my computer (with -server) Maxima on ABCL is about 10x - 20x slower than SBCL for various operations in Maxima (and using 3-4x more RAM). I don't know much about implementation details - but as far as I know, every lisp object is encapsulated in a Java object, so there is a lot of allocation going on. Compiler is also very slow (compared to SBCL). My question is, is performance ever increase dramatically (perhaps with java 7?) or is ABCL inherently slower because of the JVM? Clojure performs better than ABCL usually and is also running on JVM. I don't know about KAWA. Is JVM bytecode not flexible enough to perhaps reuse some compilation techniques of native compilers (CCL, SBCL) ? I was thinking about writing a GUI for maxima in java, using ABCL to run Maxima. This way the gui could interact directly with the lisp functions of Maxima. But the main advantage of Maxima is IMO that it can be easily extended via Lisp, which can compile to near C speed on native compilers such as SBCL and CCL. Using ABCL (if it is as slow as it is now), this would not be true and the only fast extensions to Maxima could be written in Java perhaps, which is fine, but is not portable to other CLs and Java code is verbose. Regards, Ziga
On 9/24/09 2:11 PM, Žiga Lenarčič wrote:
I've tested Maxima with ABCL. While it runs, which is rather cool, it's fairly slow! […]
My question is, is performance ever increase dramatically (perhaps with java 7?) or is ABCL inherently slower because of the JVM? Clojure performs better than ABCL usually and is also running on JVM.
But Clojure cannot run Maxima now can it?
I don't know about KAWA. Is JVM bytecode not flexible enough to perhaps reuse some compilation techniques of native compilers (CCL, SBCL) ?
Since the JVM has 8-bit opcodes and is stack based there are certainly less options to write bytecode than a modern, register-based processor. But don't forget that the bytecode itself gets transformed by the JVM to native code, so there is actually quite a bit of compilation going on "after" we compile to bytecode. We (mainly Erik?) have used Maxima as a test for understanding how to improve ABCL performance and compliance. At this point we're happy that you are wondering about speed rather than if ABCL can run Maxima. We continue to work on optimization but as a part-time volunteer effort progress is uneven. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
Žiga Lenarčič writes:
My question is, is performance ever increase dramatically (perhaps with java 7?) or is ABCL inherently slower because of the JVM?
Yes, Java7 and invokedynamic coupled with type inference in ABCL's compiler should lead to a great improvement from my understanding at least. But the changes needed would be quite substantive so I wouldn't count on it unless there's money behind it.
Clojure performs better than ABCL usually and is also running on JVM. I don't know about KAWA. Is JVM bytecode not flexible enough to perhaps reuse some compilation techniques of native compilers (CCL, SBCL) ?
Notice that ABCL is very young, and, from what I know, Clojure has one person (the original author even!) work on it full-time. -T.
Hi Ziga,
I've tested Maxima with ABCL. While it runs, which is rather cool, it's fairly slow!
It's great to see you testing ABCL! Yes, it's rather slow, that's true. We know the performance can be improved by about 30% quite easily. However, the effort to do so lies with the Maxima team: In ABCL, access to special variables is rather slow. Maxima uses global special variables with names which are later used as local non-special variables. However, since they have been declared special (globally), that code takes the "variable is special"-slowness hit too. The solution on the side of Maxima is to rename all variables which are meant to be globally special to the CL naming scheme where the name is put between asterisks.
On my computer (with -server) Maxima on ABCL is about 10x - 20x slower than SBCL for various operations in Maxima (and using 3-4x more RAM). I don't know much about implementation details - but as far as I know, every lisp object is encapsulated in a Java object, so there is a lot of allocation going on. Compiler is also very slow (compared to SBCL).
ABCL will probably always stay slower than SBCL: The JVM is a rather strict environment in which you have to play by its rules. SBCL can generate transfers of control which are not allowed by the JVM. For example, if a function has optional parameters, SBCL jumps somewhere into the middle of the function in the case where all parameters have been provided. We can't do tricks like that.
My question is, is performance ever increase dramatically (perhaps with java 7?) or is ABCL inherently slower because of the JVM? Clojure performs better than ABCL usually and is also running on JVM. I don't know about KAWA. Is JVM bytecode not flexible enough to perhaps reuse some compilation techniques of native compilers (CCL, SBCL) ?
The latter is indeed one of the problems. Also: there are a lot of micro optimizations implemented throughout the years in SBCL which make it a lot faster. And, also important: it does full-program analysis and type derivation; the most clever one of all the lisps around. This too alows for generation of a lot more efficient code. ABCL hasn't grown to a stage near any of that level.
I was thinking about writing a GUI for maxima in java, using ABCL to run Maxima. This way the gui could interact directly with the lisp functions of Maxima. But the main advantage of Maxima is IMO that it can be easily extended via Lisp, which can compile to near C speed on native compilers such as SBCL and CCL. Using ABCL (if it is as slow as it is now), this would not be true and the only fast extensions to Maxima could be written in Java perhaps, which is fine, but is not portable to other CLs and Java code is verbose.
Right. However, we really want to improve general and specific performance. So, if you have some time to analyse which functions in Maxima consume the most execution time, then maybe we can find out together how to adapt ABCL to perform better in that area? Bye, Erik.
Hi!
It's great to see you testing ABCL! Yes, it's rather slow, that's true. We know the performance can be improved by about 30% quite easily. However, the effort to do so lies with the Maxima team: In ABCL, access to special variables is rather slow. Maxima uses global special variables with names which are later used as local non-special variables. However, since they have been declared special (globally), that code takes the "variable is special"-slowness hit too.
The solution on the side of Maxima is to rename all variables which are meant to be globally special to the CL naming scheme where the name is put between asterisks.
So the naming of the variables affects the speed? Asterisk around names gives faster dynamic variables? Maxima uses too many dynamic variables in general probably - in that sense it's a badly written software.
Right. However, we really want to improve general and specific performance. So, if you have some time to analyse which functions in Maxima consume the most execution time, then maybe we can find out together how to adapt ABCL to perform better in that area?
Probably the same amount can be done on Maxima's side (to improve it's performance) as on ABCL's, but the situation is the same on both projects - it's volunteer work, so larger restructuring usually doesn't happen (at least in Maxima)... I didn't mean to 'complain' about ABCL's speed - it's already an impressive piece of software, especially considering it's open source and it hasn't got that many developers. Of course I'm aware that SBCL is derived from CMUCL which has been developed by government paid developers (and Clozure CL used to be a commercial product) so the comparison with ABCL is unfair considering man hours invested in each project. Considering ABCL's origin and it's rather intense development phase in the last year or so, it's a very 'professional' lisp implementation today. I merely wanted to know, how much 'room' is there to improve it's speed :) I was really impressed when the Maxima's testsuite ran without errors (except some small floating point differences) and even more when I opened a Swing window from within common lisp, which is (GUI) usually very hard to do using other non commercial Common Lisps! Thanks for your responses and for your work on ABCL! Regards, Ziga
On Fri, Sep 25, 2009 at 11:47 AM, Žiga Lenarčič <ziga.lenarcic@gmail.com> wrote:
So the naming of the variables affects the speed? Asterisk around names gives faster dynamic variables?
No, using asterisks for global variables only ensures that no local variable will accidentally become special (and incur in a performance hit) as the effect of a completely unrelated global variable. Example: (defvar x) (defun f (x) ...) ;x is special (let ((x ...)) ...) ;x is special (defvar *x*) (defun f (x) ...) ;x is lexical (let ((*x* ...)) ...) ;*x* is special
Maxima uses too many dynamic variables in general probably - in that sense it's a badly written software.
afaik that's because it was originally written in a dynamically scoped Lisp.
--- On Fri, 9/25/09, Alessio Stalla <alessiostalla@gmail.com> wrote:
Maxima uses too many dynamic variables in general probably - in that sense it's a badly written software.
afaik that's because it was originally written in a dynamically scoped Lisp.
I'll take this opportunity to mention that, as a long-term maintenance project, I'd like to regularize Maxima's use of special variables. First part is to use defvar instead of declare (special ...) + setq, second part is to make sure all special variables have names like *foo*. I've made some progress on the first part. Second part requires thought to distinguish uses of special variables from lexical variables with the same name; I haven't gotten started on that. I've probably omitted some important details too. If anyone would like to help with this project, I'd be glad to have your help. best Robert Dodier
participants (6)
-
Alessio Stalla -
Erik Huelsmann -
Mark Evenson -
Robert Dodier -
Tobias C. Rittweiler -
Žiga Lenarčič