armedbear-devel November 2010

armedbear-devel@common-lisp.net

12 participants
33 discussions

[armedbear-devel] [PATCH] Eliminate duplicate lookups in JHandler
by Mario Lang 09 Nov '10

09 Nov '10

Hi. A simple thing, but I spotted it during reading of the code. Index: src/org/armedbear/lisp/JHandler.java =================================================================== --- src/org/armedbear/lisp/JHandler.java (revision 12738) +++ src/org/armedbear/lisp/JHandler.java (working copy) @@ -66,21 +66,22 @@ public static void callLisp (String s, Object o, String as[], int ai[]) { if (table.containsKey(o)) { - Map<String,Entry> entryTable = (Map<String,Entry>)table.get(o); + Map<String,Entry> entryTable = (Map<String,Entry>)table.get(o); if (entryTable.containsKey(s)) { - Function f = ((Entry)entryTable.get(s)).getHandler(); - LispObject data = ((Entry)entryTable.get(s)).getData(); - Fixnum count = ((Entry)entryTable.get(s)).getCount(); - Fixnum[] lispAi = new Fixnum[ai.length]; + final Entry entry = entryTable.get(s); + final Function f = entry.getHandler(); + final LispObject data = entry.getData(); + final Fixnum count = entry.getCount(); + final Fixnum[] lispAi = new Fixnum[ai.length]; for (int i = 0; i < ai.length; i++) { lispAi[i] = Fixnum.getInstance(ai[i]); } - LispObject lispAiVector = new SimpleVector(lispAi); - SimpleString[] lispAs = new SimpleString[as.length]; + final LispObject lispAiVector = new SimpleVector(lispAi); + final SimpleString[] lispAs = new SimpleString[as.length]; for (int i = 0; i < as.length; i++) { lispAs[i] = new SimpleString(as[i]); } - LispObject lispAsVector = new SimpleVector(lispAs); + final LispObject lispAsVector = new SimpleVector(lispAs); LispObject[] args = new LispObject[] //FIXME: count -> seq_num { data, new JavaObject(o), lispAiVector, lispAsVector, internKeyword(s), count }; f.execute(args); -- CYa, ⡍⠁⠗⠊⠕

2 1

[armedbear-devel] Profiling ABCL exception handling
by Erik Huelsmann 09 Nov '10

09 Nov '10

Yesterday, I started playing around with JRockit's MissionControl tool, now that we run on JRockit. One of the (quite surprising) findings was that we generated over 1.3 million ClassNotFoundExceptions during the compilation of Maxima. With two simple changes nearly all of those have been removed. There are still quite a few exceptions associated with our loading process though, as our ZipCache seems to trigger integer parsing exceptions in URLs internally; from the names of the functions through which the exceptions are being found, it looks like Java wants to parse IPv4-dot-separated tuples as host names. Our URLs don't have hostnames most of the time: we're passing file:// URLs! Running an exception profile on Maxima running its test suite shows lots of non-local transfer of control exceptions: Return, Go and Throw: during a run of ca 10 minutes, I'm finding 5.5 million exceptions being thrown which - as MissionControl calculated for me - should mean an average of little over 12k/s. I guess that's kind of expected with our strategy for non-local transfers of control. However, if rethrows are counted as well, we can probably do better: by minimizing the number of try/catch blocks in our compiled code which reset special bindings. To illustrate what I mean, the example """ (declaim (special *x*)) (defun foo (*X*) (let ((*X* 1)) (let ((*X* 2)) 0))) """ Decompiles into (pseudo code): foo(*X*): function_setup_code reg1 = current_specials_state() try { bind_special(*X*, arg1) reg2 = current_specials_state() try { bind_special(*X*, 1) reg3 = current_specials_state() try { bind_special(*X*, 2) reg4 = integer_0 } catch (Throwable t) { reset_specials_state(reg3) rethrow t } reset_specials_state(reg3) } catch (Throwable t) { reset_specials_state(reg2) rethrow t } reset_specials_state(reg2) } catch (Throwable t) { reset_specials_state(reg1) rethrow t } reset_specials_state(reg1) return reg4 As you see, to handle a single non-local transfer of control, we throw and catch the same exception 3 times. This issue is known as ticket #90 (http://trac.common-lisp.net/armedbear/ticket/90). It would be really nice to hash through all the different situations which may need special-bindings-state to be reset with a few of you and resetting *only* those cases. I'm hoping our general performance - and the Maxima-on-ABCL performance because of its many special variables in particular. BTW: ABCL came from the situation where it tried to use that scheme, but the way it was implemented made it really hard to find out where specials were leaking - and they were. Bye, Erik.

2 3

[armedbear-devel] Performance: Analysis of options to reduce exception handlers
by Erik Huelsmann 09 Nov '10

09 Nov '10

Based on Joel's reaction regarding my profiling attempts on exception rates/exception handling in ABCL, I've been thinking how to do better in the exceptions/specials restoration department. Let me start by describing how our current exception handling/specials restoration is structured. It should be noted that the current strategy works the way it does because we were failing to restore special values all over the place. This made me decide that we should restore specials when leaving the binding block, even if that is through non-local transfer of control, with the contrived example for exception handlers as given in my mail earlier this month. Looking at the profiling results, we're now correct, but probably we're not very efficient at doing it. This is what I've been thinking: If a block binds special variables, there are 3 ways to exit the block: 1. through normal sequential execution of the byte code 2. through a function-local jump out of the block 3. through a non-local transfer of control (modelled using the ControlTransfer exception in ABCL) Case (1) is easy: we just reach the end of the binding block, at which point there is a call to "LispThread.restoreSpecialBindings(mark);". Case (2) is a little more involved: before jumping to the target location the specials are restored at the jump-location. To do this, there's a bit of logic in the compiler to find out which restore point to use. Case (3) is quite involved: at the call site an exception is thrown causing the stack to be unwound. When looking at it like this, the catch-site should restore its specials state. There's no need to do that, currently, because all the intermediate levels restore their state. There is at least one special case here: unwind-protect may be executing lisp code (which needs its specials restored to the right state) without being the final catch location. Final catch requiring forms are quite easily determined: TAGBODY (to catch GO), BLOCK (to catch RETURN) and CATCH (to catch THROW). One of the major drawbacks of this scheme is that all non-local transfers become responsible for restoring their special bindings. BLOCK and TAGBODY forms which don't serve as non-local target are not affected by this reasoning; they fall under case (2). CATCH can never be considered taking only local transfers of control as it However, the entire application may not be binding any specials at all. The overhead involves a call at the start of each block which contains non-local-transfer targets to snapshot the special bindings state and to restore it *in case of a non-local transfer*. The calls involved are markSpecialBindings() and restoreSpecialBindings(), two very short and - I think - efficient functions. So, there's a small drawback in applications which don't use special bindings much. However, the potential gain in applications which do use them is quite relevant: the example I gave before shows that a relatively simple function binding the same special variable three times will cause 3 separate specials-unwinding-exception blocks, each rethrowing the exception to the next level. Thinking about it a bit more, I realize that no additional exception blocks have to be registered. Just only the catch/restore of specials state. Looking at it this way, I propose that we stop restoring specials in finally blocks, moving part of the responsibility of having correct specials bindings to BLOCK, CATCH, TAGBODY and UNWIND-PROTECT. My change should affect both the interpreter and the compiler, to get the full benefit of efficiency. However, I thought I'd implement the LET/LET* binding optimization in the compiler first, just to see how much we benefit from this change in Maxima. Comments? Bye, Erik.

1 0

[armedbear-devel] Increased FASL size support -- was: [armedbear-cvs] r13013 - trunk/abcl/src/org/armedbear/lisp
by Erik Huelsmann 09 Nov '10

09 Nov '10

There were some reports with FASLs becoming too large -- ie generating more .cls files than can be loaded in our loader method "<fasl-name>_0" because the code overflows the 64kB method size boundary. The change below drastically reduces the size of the generated loader method, reducing the chances that people run into an issue. Of course, that can't be the final state, but it seems like a good option, as long as we don't implement the means to split this method. Implementing means to split specifically this method however, doesn't seem good enough: all of our methods have the potential of overrunning the 64kB boundary and it would be nice if we could implement general method-splitting logic instead, to be applied when methods overrun their 64kB boundary size. This commit is in our upcoming release, aleviating (but not solving) the Big FASLs problem. Bye, Erik. On Mon, Nov 8, 2010 at 11:34 PM, Erik Huelsmann <ehuelsmann(a)common-lisp.net> wrote: > Author: ehuelsmann > Date: Mon Nov 8 17:34:18 2010 > New Revision: 13013 > > Log: > Reduce the impact of the fasl-loader "_0" class's maximum size of 64kB: > this change reduces the size of pprint_0.cls from 23001 to 19380 bytes, > a saving of 18.64%, which is probably much more when counting the size > of the execute() method alone. > > Modified: > trunk/abcl/src/org/armedbear/lisp/compile-file.lisp > > Modified: trunk/abcl/src/org/armedbear/lisp/compile-file.lisp > ============================================================================== > --- trunk/abcl/src/org/armedbear/lisp/compile-file.lisp (original) > +++ trunk/abcl/src/org/armedbear/lisp/compile-file.lisp Mon Nov 8 17:34:18 2010 > @@ -679,7 +679,14 @@ > (defun generate-loader-function () > (let* ((basename (base-classname)) > (expr `(lambda (fasl-loader fn-index) > - (identity fasl-loader) ;;to avoid unused arg > + (declare (type (integer 0 256000) fn-index)) > + (identity fasl-loader) ;;to avoid unused arg > + (jvm::with-inline-code () > + (jvm::emit 'jvm::aload 1) > + (jvm::emit-invokevirtual jvm::+lisp-object+ "javaInstance" > + nil jvm::+java-object+) > + (jvm::emit-checkcast +fasl-classloader+) > + (jvm::emit 'jvm::iload 2)) > (ncase fn-index 0 ,(1- *class-number*) > ,@(loop > :for i :from 1 :to *class-number* > @@ -687,20 +694,14 @@ > (let* ((class (%format nil "org/armedbear/lisp/~A_~A" > basename i)) > (class-name (jvm::make-class-name class))) > - `(,(1- i) > - (jvm::with-inline-code () > - (jvm::emit 'jvm::aload 1) > - (jvm::emit-invokevirtual jvm::+lisp-object+ "javaInstance" > - nil jvm::+java-object+) > - (jvm::emit-checkcast +fasl-classloader+) > - (jvm::emit 'jvm::dup) > - (jvm::emit-push-constant-int ,(1- i)) > - (jvm::emit-new ,class-name) > - (jvm::emit 'jvm::dup) > - (jvm::emit-invokespecial-init ,class-name '()) > - (jvm::emit-invokevirtual +fasl-classloader+ > + `(,(1- i) > + (jvm::with-inline-code () > + (jvm::emit-new ,class-name) > + (jvm::emit 'jvm::dup) > + (jvm::emit-invokespecial-init ,class-name '()) > + (jvm::emit-invokevirtual +fasl-classloader+ > "putFunction" > - (list :int jvm::+lisp-object+) jvm::+lisp-object+) > + (list :int jvm::+lisp-object+) jvm::+lisp-object+) > (jvm::emit 'jvm::pop)) > t)))))) > (classname (fasl-loader-classname)) > > _______________________________________________ > armedbear-cvs mailing list > armedbear-cvs(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-cvs >

1 0

[armedbear-devel] Reading a UTF-8 lisp source fails.
by pjb＠informatimago.com 07 Nov '10

07 Nov '10

My sources contain some non US-ASCII characters, so they're all encoded in UTF-8. However, loading them fails: [pjb@kuiper :0.0 lisp]$ LC_CTYPE=en_US.UTF-8 abcl Armed Bear Common Lisp 0.20.0 Java 1.6.0_22 Sun Microsystems Inc. Java HotSpot(TM) Server VM Low-level initialization completed in 0.266 seconds. Startup completed in 0.726 seconds. Type ":help" for a list of available commands. CL-USER(1): (load"test-abcl-unicode.lisp") Error loading /home/pjb/src/git/public/lisp/test-abcl-unicode.lisp at line 3 (offset 54) #<THREAD "interpreter" {132021A}>: Debugger invoked on condition of type STREAM-ERROR Input value 0xffc2 is malformed while recoding with charset US-ASCII Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2): (quit) [pjb@kuiper :0.0 lisp]$ What should I do to let abcl read them as UTF-8 and not try to convert them into US-ASCII? The test file is: ------------------------------------------------------------------------ ;;;; -*- mode:lisp;coding:utf-8 -*- (defvar *test* "45°") ;;;; THE END ;;;; ------------------------------------------------------------------------ -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}.

2 1

[armedbear-devel] directory doesn't honor :wild-inferiors (#P"**/*.*")
by pjb＠informatimago.com 07 Nov '10

07 Nov '10

[pjb@kuiper :0.0 lisp]$ abcl Armed Bear Common Lisp 0.20.0 Java 1.6.0_22 Sun Microsystems Inc. Java HotSpot(TM) Server VM Low-level initialization completed in 0.266 seconds. Startup completed in 0.75 seconds. Type ":help" for a list of available commands. [1] CL-USER(9): (directory "*/*.asd") (#P"/home/pjb/src/git/public/lisp/tools/com.informatimago.common-lisp.tools.make-depends.asd" #P"/home/pjb/src/git/public/lisp/common-lisp/com.informatimago.common-lisp.asd" ...) [1] CL-USER(10): (directory #P"**/*.asd") NIL [1] CL-USER(11): -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}.

3 2

[armedbear-devel] Working toward the 0.23 release
by Erik Huelsmann 06 Nov '10

06 Nov '10

We've had a productive number of weeks since our last release. Changes include: * Running the Maxima test suite without failure [Thanks to the Maxima team for their part in this!] * CLOS thread safety fixes * Fixes to be able to run on Oracle JRockit JVM * Numerous tickets fixed * A new class file writer to build further optimizations upon So, I think we should be moving toward a release real soon now. Is there more to be added to trunk? If not, I'll be branching by the end of the weekend; otherwise, let's discuss and see what timelines to set. Everybody, thanks again for your contributions! Bye, Erik.

2 2

[armedbear-devel] Soliciting (updates to) testimonials
by Erik Huelsmann 04 Nov '10

04 Nov '10

In the interest of informing potential users about the various uses, strengths and weaknesses of ABCL, we have a Testimonials page. However, most of the testimonials don't have a date. That's rather unfortunate, because it makes it hard to judge how valid they still are: over the past three years, we had quite a bit of change in our code base. Most of the changes were driven by CLHS compliance, correctness or code cleanup (to support future maintenance). So, there's a good chance some of the older testimonials don't actually apply to current ABCL versions anymore. Just removing those older testimonials (leaving a blank page) isn't really an option for me. OTOH, leaving them up there without a date isn't really indicative of today's ABCL either. That's why I'm asking you to submit your (updated) testimonials. If you're currently using ABCL (or have been using until recently), but there's no testimonial up on the page from you, please take a quick look at the top testimonial at http://common-lisp.net/project/armedbear/testimonials.shtml (the one by David Kirkman). As you see, it doesn't have to be much, so, creating a testimonial doesn't have to take a lot of time. Taking the half hour to create a testimonial is your opportunity to donate back to the ABCL community! Thanks for your time and contributions, Erik.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

armedbear-devel November 2010