October 2009 - armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] ABCL introductory user documentation
by Paul Reiners 07 Jun '10

07 Jun '10

Some introductory user documentation I wrote on ABCL if anyone's interested: http://www.automatous-monk.com/jvmlanguages/abcl/Armed_Bear_Common_Lisp.html It's spread across a bunch of slides right now, because I first prepared it as a multi-slide presentation. I think I want to combine it all into one HTML page later this week when I get around to it. At the very least, I want to give each slide a meaningful title so that the Table of Contents is more meaningful. Any criticism, constructive or destructive, is welcome. Paul

5 11

[armedbear-devel] [BUG] EXT:ARGLIST produces bogus case-sensitive arglists
by Tobias C. Rittweiler 29 Jan '10

29 Jan '10

`arglist' in `arglist.java' calls `readObjectFromString' which uses the FASL Readtable which is case sensitive. This ain't good because this way the lambda-list keywords (&OPTIONAL etc.) are not properly read back (EXTENSIONS:|&optional|) -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

2 1

[armedbear-devel] No restarts available on error in thread
by Tobias C. Rittweiler 29 Jan '10

29 Jan '10

Evaluating the following (threads:make-thread #'(lambda () (error "foof")) :name "foof") will get you into the debugger. But there's not restart available. As it does not make sense to provide a RETURN-TO-TOPLEVEL restart for an error occuring in a thread, I think there should be a TERMINATE-THREAD restart instead. And the debugger should probably print the thread's name the error occured in. -T.

2 1

[armedbear-devel] Autoloads
by Tobias C. Rittweiler 29 Jan '10

29 Jan '10

I *always* forget what the autoload stuff is about, and when I have to add something to it. Could anyone please add a comment to Autoload.java and autoloads.lisp? -T.

2 3

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 03 Nov '09

03 Nov '09

forwarding ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Wed, Oct 28, 2009 at 11:20 PM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 9:15 PM, Erik Huelsmann <ehuels(a)gmail.com> wrote: > Last weekend, we experimented with better autoloading. It turned out > to strip roughly .4 seconds from a cold startup time of 1.7s, making > it a 25% improvement. > > However, the reason we started out with the startup time improvements > in the first place was the ABCL startup time on Google App Engine. It > turns out that our CPU usage during startup hasn't really decreased > much (as per their benchmark indicator - they can't really give an > actual figure). > > So, I asked for advice on #appengine (on freenode). Their reaction was > "we can't imagine the startup time being related to the size of the > JAR" even though Peter Graves calculated a 34% ratio between ABCL and > Clojure jar sizes and a 35% ratio between startup times - that looks > like a linear match. Their reaction continued "you're probably just > doing too much work during the init() phase." > > The init() phase is where the ABCL environment gets loaded and all > function objects get created. > > Let's assume for a second they're right. In that case we must assume > it's not I/O holding us up: it's the work the CPU must do to get us up > and running. If that's true, profiling the application should tell us > something about the bottlenecks we're running into. I happen to have > done quite a number of such profiles in the course of last week. The > conclusion which stands out is that ABCL - during the startup process > - spends ~ 40% of its time finding class constructors: the main > component of creating function objects. > > This brought me to the conclusion that our startup process could be > much faster, if we decided to delay function object creation until the > function is actually used: we would eliminate the need to construct > function objects until they're used instead of creating them when > their siblings are requested to be loaded. > > The idea is to create another Autoload derivative which will be > "installed" in the appropriate places which, when invoked, loads the > actual class from the byte array. I'm hoping this will cause a more > equally spread "initialization load". The performance hit will only be > the first call to the function: after it has been converted from the > byte array, the autoload object will remove itself from the function > call chain. > > So, how about it? Comments most welcome! I have mixed feelings about the idea. I think it's clever; but I also think we (I, at least) need more data to know if it will be actually beneficial. If the goal is speeding up startup time in a context like AppEngine - where not only Lisp, but the whole user application will be loaded from scratch from time to time - then it is critical to know how many Lisp functions a generic application uses on average (both directly and indirectly). If it turns up that, say, 50% of Lisp is commonly used, then no matter how clever an autoloading scheme you implement, you'll cut loading times only by roughly 50% at best. If getting constructors through reflection is really the bottleneck, and if we determine that using new instead of reflection is significantly faster (from a quick test of mine, it seems it *really* is [1]), then it might be sensible to avoid reflection altogether and devise another scheme. For example, the compiler-generated class X could contain in its static initialization block the equivalent of something like Lisp.someThreadLocal.set(new X()) and loadCompiledFunction or what it is could just fetch the instance from the threadlocal; not very elegant, but if it speeds things up... Alessio [1] this is the astounding result on a couple of runs on 50000 iterations (test files attached): REFLECTION: 16262373155 NEW: 84267527 % SLOWER: 19298 REFLECTION: 15917190176 NEW: 103681915 % SLOWER: 15351 REFLECTION: 15838714133 NEW: 77235481 % SLOWER: 20507 (times in ns) i.e. reflection as we use it is roughly 150-200 times slower than new and that's on a very simple class with no superclasses and a single constructor! The test might be wrong as I wrote it quickly and it's quite tricky. It uses the very same classloader of abcl, though (copy-pasted).

3 3

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 29 Oct '09

29 Oct '09

forwarding ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Wed, Oct 28, 2009 at 11:34 PM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 11:20 PM, Alessio Stalla <alessiostalla(a)gmail.com> wrote: > The test might be wrong It was, indeed, wrong. It measured class loading time too for reflection, but that can't be eliminated, no matter what instantiation technique you use. New results on 50000 iterations: REFLECTION: 9660480097 NEW: 88353767 % SLOWER: 10933 REFLECTION: 9602410291 NEW: 78459331 % SLOWER: 12238 REFLECTION: 9925574038 NEW: 86923015 % SLOWER: 11418 ~100 times slower is still impressive. Ale

1 1

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 29 Oct '09

29 Oct '09

Sorry, we seem to be having this discussion on the wrong mailing list. Forwarding. ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Thu, Oct 29, 2009 at 9:36 AM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 11:34 PM, Alessio Stalla <alessiostalla(a)gmail.com> wrote: > On Wed, Oct 28, 2009 at 11:20 PM, Alessio Stalla > <alessiostalla(a)gmail.com> wrote: >> The test might be wrong > > It was, indeed, wrong. It measured class loading time too for > reflection, but that can't be eliminated, no matter what instantiation > technique you use. > > New results on 50000 iterations: > > REFLECTION: 9660480097 > NEW: 88353767 > % SLOWER: 10933 > > REFLECTION: 9602410291 > NEW: 78459331 > % SLOWER: 12238 > > REFLECTION: 9925574038 > NEW: 86923015 > % SLOWER: 11418 > > ~100 times slower is still impressive. Incidentally, if those figures are right, it means that class loading from byte array is very costly too, and Erik's idea is likely to improve performance significantly if the application never uses a relatively large part of Lisp.

1 0

Re: [armedbear-devel] [armedbear-cvs] r12226 - trunk/abcl/src/org/armedbear/lisp
by Erik Huelsmann 27 Oct '09

27 Oct '09

> Modified: trunk/abcl/src/org/armedbear/lisp/compiler-pass2.lisp > ============================================================================== > --- trunk/abcl/src/org/armedbear/lisp/compiler-pass2.lisp (original) > +++ trunk/abcl/src/org/armedbear/lisp/compiler-pass2.lisp Sun Oct 25 18:35:52 2009 > @@ -1948,6 +1948,8 @@ > (when (plusp (length output)) > output))) > > +(defvar *declare-inline* nil) > + > (defmacro declare-with-hashtable (declared-item hashtable hashtable-var > item-var &body body) > `(let* ((,hashtable-var ,hashtable) > @@ -1970,19 +1972,19 @@ > (declare-object symbol +lisp-symbol+ > +lisp-symbol-class+)))) > (t > - (let ((*code* *static-code*) > - (s (sanitize symbol))) > - (setf g (symbol-name (gensym "SYM"))) > - (when s > - (setf g (concatenate 'string g "_" s))) > - (declare-field g +lisp-symbol+ +field-access-private+) > - (emit 'ldc (pool-string (symbol-name symbol))) > - (emit 'ldc (pool-string (package-name (symbol-package symbol)))) > - (emit-invokestatic +lisp-class+ "internInPackage" > - (list +java-string+ +java-string+) +lisp-symbol+) > - (emit 'putstatic *this-class* g +lisp-symbol+) > - (setf *static-code* *code*) > - (setf (gethash symbol ht) g)))))) > + (let ((*code* *static-code*) > + (s (sanitize symbol))) > + (setf g (symbol-name (gensym "SYM"))) > + (when s > + (setf g (concatenate 'string g "_" s))) > + (declare-field g +lisp-symbol+ +field-access-private+) > + (emit 'ldc (pool-string (symbol-name symbol))) > + (emit 'ldc (pool-string (package-name (symbol-package symbol)))) > + (emit-invokestatic +lisp-class+ "internInPackage" > + (list +java-string+ +java-string+) +lisp-symbol+) > + (emit 'putstatic *this-class* g +lisp-symbol+) > + (setf *static-code* *code*) > + (setf (gethash symbol ht) g)))))) There's a thinko in the above hunk: it only re-indents, that was intentional. However, it assumes the package to intern into exist. When *declare-inline* is non-NIL, there's a fair chance the package hasn't been created at *static-code* time though. This means the symbol creation will need to happen in-line too. I'll fix this later today. Bye, Erik.

1 0

[armedbear-devel] Change the default action of 'build.xml' to 'abcl.clean abcl.wrapper'?
by Mark Evenson 26 Oct '09

26 Oct '09

[Including armedbear-devel@ for potential wider dicussion.] On 10/26/09 1:05 PM, Erik Huelsmann wrote: […] > > BTW: I'm always doing complete builds; are you too? Yeah, in fact I think the much more common use case of the build system is 'ant abcl.clean abcl.wrapper'. I almost always want a full build from scratch, which is now taking slightly under two minutes for my MacBook Pro laptop. A couple years ago, a full build took something like five minutes from what I remember, so there was more of a need for incremental builds. What's your usage pattern like? Maybe I should make 'abcl.clean abcl.wrapper' the default target, where the old behavior could be re-restablished via setting an Ant property. This is contrary to the behavior of most build systems that I have experienced, but since we could write a banner message like "Starting build from scratch; set 'abcl.build.incremental' in 'build.properties' to enable incremental builds in the future." Thoughts? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."

2 1

[armedbear-devel] Binary FASL support
by Erik Huelsmann 26 Oct '09

26 Oct '09

As the result of an attempt to reduce our startup time, I implemented binary FASL support: a compiled lisp function (= Java class) which gets executed instead of the interpreted forms which used to constitute the texty fasl format. The texty format is still there, but in case of a binary fasl only contains 2 forms to be evaluated, the rest will be in the compiled function. I don't feel like we achieved much: my startup times used to be 0.4 s and 1.6 s and that doesn't seem to have changed. Bummer. However, that's probably to be expected, because I was just guessing where our performance issue would be: I want to know how much of the 0.4 and 1.6 seconds are actually spent waiting on I/O happening. In case we're waiting big time on the data coming from the JAR, maybe there's an optimization possible to the way we create/read the JAR file. Any help on the subject of reducing our startup time would be most welcome! Bye, Erik.

2 1