- armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] ABCL basics
by dusan lukic 05 Nov '09

05 Nov '09

Hi all, with your help, I managed to create abcl.jar file, and now I need some basic knowledge of using it. I would like to know how to compile Lisp code (in a form of String or text file) and call it from Java code. I would also like to get some basic idea of which classes are defined in the abcl.jar file, and what is their purpose and what can I do with them. Furthermore, I would like to obtain some abcl help (it can be online, too), which contain descriptions of abcl classes and their methods, fields, stuff like that. Can anyone point me to some documentation or give some guidelines? Thaks.

2 1

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 03 Nov '09

03 Nov '09

forwarding ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Wed, Oct 28, 2009 at 11:20 PM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 9:15 PM, Erik Huelsmann <ehuels(a)gmail.com> wrote: > Last weekend, we experimented with better autoloading. It turned out > to strip roughly .4 seconds from a cold startup time of 1.7s, making > it a 25% improvement. > > However, the reason we started out with the startup time improvements > in the first place was the ABCL startup time on Google App Engine. It > turns out that our CPU usage during startup hasn't really decreased > much (as per their benchmark indicator - they can't really give an > actual figure). > > So, I asked for advice on #appengine (on freenode). Their reaction was > "we can't imagine the startup time being related to the size of the > JAR" even though Peter Graves calculated a 34% ratio between ABCL and > Clojure jar sizes and a 35% ratio between startup times - that looks > like a linear match. Their reaction continued "you're probably just > doing too much work during the init() phase." > > The init() phase is where the ABCL environment gets loaded and all > function objects get created. > > Let's assume for a second they're right. In that case we must assume > it's not I/O holding us up: it's the work the CPU must do to get us up > and running. If that's true, profiling the application should tell us > something about the bottlenecks we're running into. I happen to have > done quite a number of such profiles in the course of last week. The > conclusion which stands out is that ABCL - during the startup process > - spends ~ 40% of its time finding class constructors: the main > component of creating function objects. > > This brought me to the conclusion that our startup process could be > much faster, if we decided to delay function object creation until the > function is actually used: we would eliminate the need to construct > function objects until they're used instead of creating them when > their siblings are requested to be loaded. > > The idea is to create another Autoload derivative which will be > "installed" in the appropriate places which, when invoked, loads the > actual class from the byte array. I'm hoping this will cause a more > equally spread "initialization load". The performance hit will only be > the first call to the function: after it has been converted from the > byte array, the autoload object will remove itself from the function > call chain. > > So, how about it? Comments most welcome! I have mixed feelings about the idea. I think it's clever; but I also think we (I, at least) need more data to know if it will be actually beneficial. If the goal is speeding up startup time in a context like AppEngine - where not only Lisp, but the whole user application will be loaded from scratch from time to time - then it is critical to know how many Lisp functions a generic application uses on average (both directly and indirectly). If it turns up that, say, 50% of Lisp is commonly used, then no matter how clever an autoloading scheme you implement, you'll cut loading times only by roughly 50% at best. If getting constructors through reflection is really the bottleneck, and if we determine that using new instead of reflection is significantly faster (from a quick test of mine, it seems it *really* is [1]), then it might be sensible to avoid reflection altogether and devise another scheme. For example, the compiler-generated class X could contain in its static initialization block the equivalent of something like Lisp.someThreadLocal.set(new X()) and loadCompiledFunction or what it is could just fetch the instance from the threadlocal; not very elegant, but if it speeds things up... Alessio [1] this is the astounding result on a couple of runs on 50000 iterations (test files attached): REFLECTION: 16262373155 NEW: 84267527 % SLOWER: 19298 REFLECTION: 15917190176 NEW: 103681915 % SLOWER: 15351 REFLECTION: 15838714133 NEW: 77235481 % SLOWER: 20507 (times in ns) i.e. reflection as we use it is roughly 150-200 times slower than new and that's on a very simple class with no superclasses and a single constructor! The test might be wrong as I wrote it quickly and it's quite tricky. It uses the very same classloader of abcl, though (copy-pasted).

3 3

[armedbear-devel] Focusing 0.18 efforts
by Erik Huelsmann 03 Nov '09

03 Nov '09

Today, I branched for 0.17 (to be released soon), meaning that trunk has become free for 0.18 development. Personally, I found there are a number of items open any one of which could become part of our 0.18 effort. These are the points I identified: * Investigate and improve startup performance * CLOS conformance improvements (including define-method-combination support?) * Cleanup of Load.java - which would allow loading compressed fasls off streams * Gray stream compatibility finalization [we're 99% compatible, but FUNDAMENTAL-STREAM should be recognized as a subtype of STREAM] * Some random other stuff that comes along Do you have your own, or opinions on the points I mentioned above? Let's hear it! Bye, Erik.

2 3

[armedbear-devel] One more change for 0.17: FASL format update
by Erik Huelsmann 03 Nov '09

03 Nov '09

During the coming into existence of 0.17, there was an API change with respect to binding specials. This api change allows quick local access to the binding generated. However, the function for which the api signature is changed, is part of our generated .cls files (FASLs). Hence, I conclude we need a FASL version increment before 0.17 - which hasn't happened yet. Regards, Erik.

1 0

[armedbear-devel] Fw: [jvm-l] Re: Some Array vs GETFILED access times
by logicmoo＠gmail.com 01 Nov '09

01 Nov '09

----- Original Message ----- From: "Charles Oliver Nutter" <headius(a)headius.com> To: <jvm-languages(a)googlegroups.com> Sent: Sunday, November 01, 2009 8:47 PM Subject: [jvm-l] Re: Some Array vs GETFILED access times None of your results are too surprising to me. In JRuby, we moved several structures from encapsulating an array to using differently-sized "slotted" objects, with performance registering much better as a result. Some of that was due to the lack of boundschecks on the arrays, but another portion was probably due to the reduced size of an object + reference versus object + reference + array. Some comments inline below. On Sun, Nov 1, 2009 at 7:48 PM, <logicmoo(a)gmail.com> wrote: > public class ArrayVsClass { > > public static void main(String[] args) { > long lVectorStartTime = System.currentTimeMillis(); > int iterations = Integer.MAX_VALUE; > while (iterations-- > 0) { > // int iterations2 = 10; > //while (iterations2-- > 0) > { > //testArray(); // ARRAY > // vs > //testGETFIELD(); // GETFIELD > // vs > testIBean(); // INVOKE INTERFACE > // vs > //testBean(); // INVOKE VIRTUAL > // vs > //testABean(); // INVOKE VIRTUAL POSSIBLY THROW > // vs > //testSlots(); // INVOKE FOR AVALUE > } > } > long lVectorRunTime = System.currentTimeMillis() - lVectorStartTime; > System.out.println("Bench time: " + lVectorRunTime); > > } Because of optimization effects, you should try running them all together in the same benchmark in varying orders. Short benchmarks like these can skew actual results because only a small subset of the full code needs to be considered for optimization. > // SLOTS time: 33157,33250,33156 > public static void testSlots() { > ClassWithSlots oneSlot = new ClassWithSlots(6); ... > // Array time: 18438,18437,18422 > public static void testArray() { > final long[] accessArray = new long[] { 6 }; Not too surprising; you're paying the cost of the array plus the cost of the virtual invocation. So even after inlining, you've got something like boundscheck + deref + virtual call. > // GETFIELD time: 14688,14531,14453 > public static void testGETFIELD() { > ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); ... > // INVOKE VIRTUAL time: 14750,14594,14719 > public static void testBean() { > ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); This is exactly the pattern we use in JRuby for heap-based scopes. We have from ZeroVarDynamicScope up to FourVarDynamicScope and then it falls over into an array-based version. Because we can statically tell how many variable slots we'll need in most Ruby scopes, this ended up being a big perf improvement for us. > // INVOKE INTERFACE time: 14469,14610,14859 > public static void testIBean() { > IBeanWithOneSlot oneSlot = new ClassWithOneSlot(6); > > int iterations = Integer.MAX_VALUE; > long result = 0; > while (iterations-- > 0) { > result += oneSlot.getValue(); > } > } invokeinterface ends up as fast as invokevirtual once it's been inlined, so this is no surprise. > // INVOKE VIRTUAL POSSIBLY THROW time: 14641,14594,14547 > public static void testABean() { > AClassWithOneSlot oneSlot = new ClassWithOneSlot(6); Exception-handling paths not followed do not impact performance, so this is also not surprising. Try having one out of the N invocations trigger the exception and watch the perf change drastically from then on. - Charlie --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JVM Languages" group. To post to this group, send email to jvm-languages(a)googlegroups.com To unsubscribe from this group, send email to jvm-languages+unsubscribe(a)googlegroups.com For more options, visit this group at http://groups.google.com/group/jvm-languages?hl=en -~----------~----~----~----~------~----~------~--~---

1 0

[armedbear-devel] Some Array vs GETFILED access times
by logicmoo＠gmail.com 01 Nov '09

01 Nov '09

ABCL, For implementing MOP-like objects these times are something to consider. What I draw from it is when not messing with arrays (array bounds checking). Every case is better than. JVM-L, Do these tests with times make sense? package ArrayTests; public class ArrayVsClass { public static void main(String[] args) { long lVectorStartTime = System.currentTimeMillis(); int iterations = Integer.MAX_VALUE; while (iterations-- > 0) { // int iterations2 = 10; //while (iterations2-- > 0) { //testArray(); // ARRAY // vs //testGETFIELD(); // GETFIELD // vs testIBean(); // INVOKE INTERFACE // vs //testBean(); // INVOKE VIRTUAL // vs //testABean(); // INVOKE VIRTUAL POSSIBLY THROW // vs //testSlots(); // INVOKE FOR AVALUE } } long lVectorRunTime = System.currentTimeMillis() - lVectorStartTime; System.out.println("Bench time: " + lVectorRunTime); } // SLOTS time: 33157,33250,33156 public static void testSlots() { ClassWithSlots oneSlot = new ClassWithSlots(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } // Array time: 18438,18437,18422 public static void testArray() { final long[] accessArray = new long[] { 6 }; int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += accessArray[0]; } } // GETFIELD time: 14688,14531,14453 public static void testGETFIELD() { ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.slot; } } // INVOKE VIRTUAL time: 14750,14594,14719 public static void testBean() { ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } // INVOKE INTERFACE time: 14469,14610,14859 public static void testIBean() { IBeanWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } // INVOKE VIRTUAL POSSIBLY THROW time: 14641,14594,14547 public static void testABean() { AClassWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } static interface IBeanWithOneSlot { public long getValue(); } static class ClassWithOneSlot extends AClassWithOneSlot implements IBeanWithOneSlot { final public long slot; ClassWithOneSlot(long s) { slot = s; } @Override final public long getValue() { return slot; } } static class ClassWithSlots { final public long[] slots = new long[1]; ClassWithSlots(long s) { slots[0] = s; } final public long getValue() { return slots[0]; } } static abstract class AClassWithOneSlot implements IBeanWithOneSlot { @Override public long getValue() { throw new NullPointerException(); } } }

1 0

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 29 Oct '09

29 Oct '09

forwarding ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Wed, Oct 28, 2009 at 11:34 PM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 11:20 PM, Alessio Stalla <alessiostalla(a)gmail.com> wrote: > The test might be wrong It was, indeed, wrong. It measured class loading time too for reflection, but that can't be eliminated, no matter what instantiation technique you use. New results on 50000 iterations: REFLECTION: 9660480097 NEW: 88353767 % SLOWER: 10933 REFLECTION: 9602410291 NEW: 78459331 % SLOWER: 12238 REFLECTION: 9925574038 NEW: 86923015 % SLOWER: 11418 ~100 times slower is still impressive. Ale

1 1

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 29 Oct '09

29 Oct '09

Sorry, we seem to be having this discussion on the wrong mailing list. Forwarding. ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Thu, Oct 29, 2009 at 9:36 AM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 11:34 PM, Alessio Stalla <alessiostalla(a)gmail.com> wrote: > On Wed, Oct 28, 2009 at 11:20 PM, Alessio Stalla > <alessiostalla(a)gmail.com> wrote: >> The test might be wrong > > It was, indeed, wrong. It measured class loading time too for > reflection, but that can't be eliminated, no matter what instantiation > technique you use. > > New results on 50000 iterations: > > REFLECTION: 9660480097 > NEW: 88353767 > % SLOWER: 10933 > > REFLECTION: 9602410291 > NEW: 78459331 > % SLOWER: 12238 > > REFLECTION: 9925574038 > NEW: 86923015 > % SLOWER: 11418 > > ~100 times slower is still impressive. Incidentally, if those figures are right, it means that class loading from byte array is very costly too, and Erik's idea is likely to improve performance significantly if the application never uses a relatively large part of Lisp.

1 0

Re: [armedbear-devel] [armedbear-cvs] r12226 - trunk/abcl/src/org/armedbear/lisp
by Erik Huelsmann 27 Oct '09

27 Oct '09

> Modified: trunk/abcl/src/org/armedbear/lisp/compiler-pass2.lisp > ============================================================================== > --- trunk/abcl/src/org/armedbear/lisp/compiler-pass2.lisp (original) > +++ trunk/abcl/src/org/armedbear/lisp/compiler-pass2.lisp Sun Oct 25 18:35:52 2009 > @@ -1948,6 +1948,8 @@ > (when (plusp (length output)) > output))) > > +(defvar *declare-inline* nil) > + > (defmacro declare-with-hashtable (declared-item hashtable hashtable-var > item-var &body body) > `(let* ((,hashtable-var ,hashtable) > @@ -1970,19 +1972,19 @@ > (declare-object symbol +lisp-symbol+ > +lisp-symbol-class+)))) > (t > - (let ((*code* *static-code*) > - (s (sanitize symbol))) > - (setf g (symbol-name (gensym "SYM"))) > - (when s > - (setf g (concatenate 'string g "_" s))) > - (declare-field g +lisp-symbol+ +field-access-private+) > - (emit 'ldc (pool-string (symbol-name symbol))) > - (emit 'ldc (pool-string (package-name (symbol-package symbol)))) > - (emit-invokestatic +lisp-class+ "internInPackage" > - (list +java-string+ +java-string+) +lisp-symbol+) > - (emit 'putstatic *this-class* g +lisp-symbol+) > - (setf *static-code* *code*) > - (setf (gethash symbol ht) g)))))) > + (let ((*code* *static-code*) > + (s (sanitize symbol))) > + (setf g (symbol-name (gensym "SYM"))) > + (when s > + (setf g (concatenate 'string g "_" s))) > + (declare-field g +lisp-symbol+ +field-access-private+) > + (emit 'ldc (pool-string (symbol-name symbol))) > + (emit 'ldc (pool-string (package-name (symbol-package symbol)))) > + (emit-invokestatic +lisp-class+ "internInPackage" > + (list +java-string+ +java-string+) +lisp-symbol+) > + (emit 'putstatic *this-class* g +lisp-symbol+) > + (setf *static-code* *code*) > + (setf (gethash symbol ht) g)))))) There's a thinko in the above hunk: it only re-indents, that was intentional. However, it assumes the package to intern into exist. When *declare-inline* is non-NIL, there's a fair chance the package hasn't been created at *static-code* time though. This means the symbol creation will need to happen in-line too. I'll fix this later today. Bye, Erik.

1 0

[armedbear-devel] Change the default action of 'build.xml' to 'abcl.clean abcl.wrapper'?
by Mark Evenson 26 Oct '09

26 Oct '09

[Including armedbear-devel@ for potential wider dicussion.] On 10/26/09 1:05 PM, Erik Huelsmann wrote: […] > > BTW: I'm always doing complete builds; are you too? Yeah, in fact I think the much more common use case of the build system is 'ant abcl.clean abcl.wrapper'. I almost always want a full build from scratch, which is now taking slightly under two minutes for my MacBook Pro laptop. A couple years ago, a full build took something like five minutes from what I remember, so there was more of a need for incremental builds. What's your usage pattern like? Maybe I should make 'abcl.clean abcl.wrapper' the default target, where the old behavior could be re-restablished via setting an Ant property. This is contrary to the behavior of most build systems that I have experienced, but since we could write a banner message like "Starting build from scratch; set 'abcl.build.incremental' in 'build.properties' to enable incremental builds in the future." Thoughts? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."

2 1