November 2010 - armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] Significant increase in build time
by Blake McBride 16 Nov '10

16 Nov '10

Greetings, I hadn't built ABCL in a little while so I checked out the latest version today and built it. It seemed to be significantly slower than before so I decided to investigate. This is what I found. In the past I could build ABCL in 2:43. It now takes me 4:40. That (IMO) represents a pretty significant change in build time. I did a binary search and discovered that all of the change occurred at revision 12918 - Generic Class File Branch Merge. In general, I could't care less about the build time unless it is indicative of a problem that could rear its head in my application. Where is that time being spent? Is there a change in runtime? Loading? Compiling? I'd be real interested in this. Thanks. Blake McBride

4 10

[armedbear-devel] cl+ssl problem
by Cyrus Harmon 15 Nov '10

15 Nov '10

Dear ABCL Developers, Attempting to asdf:load cl+ssl gives the following error: Loadable FASL not found for '/Users/sly/.cache/common-lisp/armedbear-0.23.0-dev-fasl36-darwin-java-1.6/Users/sly/src/clbuild/source/cl+ssl/package.abcl' in 'jar:file:/Users/sly/.cache/common-lisp/armedbear-0.23.0-dev-fasl36-darwin-java-1.6/Users/sly/src/clbuild/source/cl ssl/package.abcl!/package._' [Condition of type FILE-ERROR] A trivial rename of the .asd file (and the contained system) to, say, clssl, allows ABCL to get past this, so it looks like the + in the file name (or ASDF system name) is causing the problem. Of course the trivial fix doesn't really fix things as there's still a CFFI problem, but that's for another email.... thanks, Cyrus

3 3

[armedbear-devel] Java FFI performance enhancement idea: LRU cache
by Erik Huelsmann 12 Nov '10

12 Nov '10

Hi, Today, on IRC, I was approached by 'pmd' whether we have weak hash tables. We really don't, but he was actually looking for a way to memoize Java methods to be called with jmethod/jcall. Since we do kind of 'dynamic dispatch' (which can be a costly operation), I thought it would be a good idea to add this memoization to our Java.java file using an LRU cache to clear old cache elements and maybe with an interface to clear the cache or resize it. Comments? Regards, Erik.

2 1

Re: [armedbear-devel] [armedbear-cvs] r13017 - trunk/abcl/src/org/armedbear/lisp
by Ville Voutilainen 11 Nov '10

11 Nov '10

On 11 November 2010 00:23, Mark Evenson <mevenson(a)common-lisp.net> wrote: > Log: > Check type in MAKE-PATHNAME for :DIRECTORY components. > + error(new TypeError("DIRECTORY argument not a string, list of strings, nil, :WILD, or :UNSPECIFIC.", value, NIL)); Where's :WILD-INFERIORS?

3 2

[armedbear-devel] compilation successfull, but launch fails with the infamous "Failed to find loadable system file 'autoloads' in boot classpath."
by pjb＠informatimago.com 09 Nov '10

09 Nov '10

Hi! I tried to compile abcl-0.22.0 on gentoo x86-64 with SBCL 1.0.19-gentoo and /opt/sun-jdk-1.6.0.22/. It looks like it went well, but I get "Failed to find loadable system file 'autoloads' in boot classpath." when trying to launch abcl. I could find some pages with this diagnostic on the web, but no comment or solution. [pjb@kuiper :0.0 abcl-0.22.0]$ Linux kuiper 2.6.34-xen-r3-kvm-nvidia-joy-c7 #3 SMP Fri Oct 8 12:01:59 CEST 2010 x86_64 Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz GenuineIntel GNU/Linux [pjb@kuiper :0.0 abcl-0.22.0]$ tar zxf /data/src/languages/abcl-src-0.22.0.tar.gz [pjb@kuiper :0.0 abcl-0.22.0]$ mv abcl-src-0.22.0/ src [pjb@kuiper :0.0 abcl-0.22.0]$ cd src [pjb@kuiper :0.0 src]$ cp customizations.lisp.in customizations.lisp [pjb@kuiper :0.0 src]$ ./build-from-lisp.sh sbcl bash: ./build-from-lisp.sh: Permission denied [pjb@kuiper :0.0 src]$ chmod 755 ./build-from-lisp.sh [pjb@kuiper :0.0 src]$ ./build-from-lisp.sh sbcl This is SBCL 1.0.19-gentoo, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. ; loading system definition from ; /usr/share/common-lisp/systems/asdf-binary-locations.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM ASDF-BINARY-LOCATIONS {1002BFA1B1}> as ; ASDF-BINARY-LOCATIONS ; in: LAMBDA NIL ; (LET* ((STRING (NAMESTRING PATHNAME)) (BUILD-ABCL::LEN (LENGTH STRING))) ; (WHEN (POSITION #\ STRING) ; (SETF STRING ; (CONCATENATE 'STRING "\"" (BUILD-ABCL::COMP STRING #\\) "\""))) ; STRING) ; ; caught STYLE-WARNING: ; The variable LEN is defined but never used. ; ; compilation unit finished ; caught 1 STYLE-WARNING condition Platform: Linux JDK: /opt/sun-jdk-1.6.0.22/ Java compiler: /opt/sun-jdk-1.6.0.22/bin/javac Compiler options: -g Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: JavaObject.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. Armed Bear Common Lisp 0.22.0 Java 1.6.0_22 Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.228 seconds. Startup completed in 2.617 seconds. #<THREAD "interpreter" {7433B121}>: Debugger invoked on condition of type ERROR Failed to find loadable system file 'system' in boot classpath. [1] CL-USER(1): Build completed successfully in 7.457 seconds. [pjb@kuiper :0.0 src]$ ./abcl --eval '(lisp-implementation-version)' Armed Bear Common Lisp 0.22.0 Java 1.6.0_22 Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.223 seconds. java.lang.Error at org.armedbear.lisp.Primitives$pf_error.execute(Primitives.java:1570) at org.armedbear.lisp.Primitive.execute(Primitive.java:113) at org.armedbear.lisp.Symbol.execute(Symbol.java:776) at org.armedbear.lisp.Lisp.error(Lisp.java:354) at org.armedbear.lisp.Load.loadSystemFile(Load.java:272) at org.armedbear.lisp.Load$load_system_file.execute(Load.java:691) at org.armedbear.lisp.LispThread.execute(LispThread.java:568) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:520) at org.armedbear.lisp.Lisp.eval(Lisp.java:485) at org.armedbear.lisp.Load.loadStream(Load.java:566) at org.armedbear.lisp.Load.loadFileFromStream(Load.java:534) at org.armedbear.lisp.Load.loadFileFromStream(Load.java:424) at org.armedbear.lisp.Load.loadSystemFile(Load.java:323) at org.armedbear.lisp.Interpreter.initializeLisp(Interpreter.java:163) at org.armedbear.lisp.Interpreter.createDefaultInstance(Interpreter.java:94) at org.armedbear.lisp.Main$1.run(Main.java:48) at java.lang.Thread.run(Thread.java:662) ERROR placeholder called with arguments: #<ERROR {647109C4}> "Failed to find loadable system file 'autoloads' in boot classpath." [pjb@kuiper :0.0 src]$ -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}.

2 1

[armedbear-devel] [PATCH] Eliminate duplicate lookups in JHandler
by Mario Lang 09 Nov '10

09 Nov '10

Hi. A simple thing, but I spotted it during reading of the code. Index: src/org/armedbear/lisp/JHandler.java =================================================================== --- src/org/armedbear/lisp/JHandler.java (revision 12738) +++ src/org/armedbear/lisp/JHandler.java (working copy) @@ -66,21 +66,22 @@ public static void callLisp (String s, Object o, String as[], int ai[]) { if (table.containsKey(o)) { - Map<String,Entry> entryTable = (Map<String,Entry>)table.get(o); + Map<String,Entry> entryTable = (Map<String,Entry>)table.get(o); if (entryTable.containsKey(s)) { - Function f = ((Entry)entryTable.get(s)).getHandler(); - LispObject data = ((Entry)entryTable.get(s)).getData(); - Fixnum count = ((Entry)entryTable.get(s)).getCount(); - Fixnum[] lispAi = new Fixnum[ai.length]; + final Entry entry = entryTable.get(s); + final Function f = entry.getHandler(); + final LispObject data = entry.getData(); + final Fixnum count = entry.getCount(); + final Fixnum[] lispAi = new Fixnum[ai.length]; for (int i = 0; i < ai.length; i++) { lispAi[i] = Fixnum.getInstance(ai[i]); } - LispObject lispAiVector = new SimpleVector(lispAi); - SimpleString[] lispAs = new SimpleString[as.length]; + final LispObject lispAiVector = new SimpleVector(lispAi); + final SimpleString[] lispAs = new SimpleString[as.length]; for (int i = 0; i < as.length; i++) { lispAs[i] = new SimpleString(as[i]); } - LispObject lispAsVector = new SimpleVector(lispAs); + final LispObject lispAsVector = new SimpleVector(lispAs); LispObject[] args = new LispObject[] //FIXME: count -> seq_num { data, new JavaObject(o), lispAiVector, lispAsVector, internKeyword(s), count }; f.execute(args); -- CYa, ⡍⠁⠗⠊⠕

2 1

[armedbear-devel] Profiling ABCL exception handling
by Erik Huelsmann 09 Nov '10

09 Nov '10

Yesterday, I started playing around with JRockit's MissionControl tool, now that we run on JRockit. One of the (quite surprising) findings was that we generated over 1.3 million ClassNotFoundExceptions during the compilation of Maxima. With two simple changes nearly all of those have been removed. There are still quite a few exceptions associated with our loading process though, as our ZipCache seems to trigger integer parsing exceptions in URLs internally; from the names of the functions through which the exceptions are being found, it looks like Java wants to parse IPv4-dot-separated tuples as host names. Our URLs don't have hostnames most of the time: we're passing file:// URLs! Running an exception profile on Maxima running its test suite shows lots of non-local transfer of control exceptions: Return, Go and Throw: during a run of ca 10 minutes, I'm finding 5.5 million exceptions being thrown which - as MissionControl calculated for me - should mean an average of little over 12k/s. I guess that's kind of expected with our strategy for non-local transfers of control. However, if rethrows are counted as well, we can probably do better: by minimizing the number of try/catch blocks in our compiled code which reset special bindings. To illustrate what I mean, the example """ (declaim (special *x*)) (defun foo (*X*) (let ((*X* 1)) (let ((*X* 2)) 0))) """ Decompiles into (pseudo code): foo(*X*): function_setup_code reg1 = current_specials_state() try { bind_special(*X*, arg1) reg2 = current_specials_state() try { bind_special(*X*, 1) reg3 = current_specials_state() try { bind_special(*X*, 2) reg4 = integer_0 } catch (Throwable t) { reset_specials_state(reg3) rethrow t } reset_specials_state(reg3) } catch (Throwable t) { reset_specials_state(reg2) rethrow t } reset_specials_state(reg2) } catch (Throwable t) { reset_specials_state(reg1) rethrow t } reset_specials_state(reg1) return reg4 As you see, to handle a single non-local transfer of control, we throw and catch the same exception 3 times. This issue is known as ticket #90 (http://trac.common-lisp.net/armedbear/ticket/90) It would be really nice to hash through all the different situations which may need special-bindings-state to be reset with a few of you and resetting *only* those cases. I'm hoping our general performance - and the Maxima-on-ABCL performance because of its many special variables in particular. BTW: ABCL came from the situation where it tried to use that scheme, but the way it was implemented made it really hard to find out where specials were leaking - and they were. Bye, Erik.

2 3

[armedbear-devel] Performance: Analysis of options to reduce exception handlers
by Erik Huelsmann 09 Nov '10

09 Nov '10

Based on Joel's reaction regarding my profiling attempts on exception rates/exception handling in ABCL, I've been thinking how to do better in the exceptions/specials restoration department. Let me start by describing how our current exception handling/specials restoration is structured. It should be noted that the current strategy works the way it does because we were failing to restore special values all over the place. This made me decide that we should restore specials when leaving the binding block, even if that is through non-local transfer of control, with the contrived example for exception handlers as given in my mail earlier this month. Looking at the profiling results, we're now correct, but probably we're not very efficient at doing it. This is what I've been thinking: If a block binds special variables, there are 3 ways to exit the block: 1. through normal sequential execution of the byte code 2. through a function-local jump out of the block 3. through a non-local transfer of control (modelled using the ControlTransfer exception in ABCL) Case (1) is easy: we just reach the end of the binding block, at which point there is a call to "LispThread.restoreSpecialBindings(mark);". Case (2) is a little more involved: before jumping to the target location the specials are restored at the jump-location. To do this, there's a bit of logic in the compiler to find out which restore point to use. Case (3) is quite involved: at the call site an exception is thrown causing the stack to be unwound. When looking at it like this, the catch-site should restore its specials state. There's no need to do that, currently, because all the intermediate levels restore their state. There is at least one special case here: unwind-protect may be executing lisp code (which needs its specials restored to the right state) without being the final catch location. Final catch requiring forms are quite easily determined: TAGBODY (to catch GO), BLOCK (to catch RETURN) and CATCH (to catch THROW). One of the major drawbacks of this scheme is that all non-local transfers become responsible for restoring their special bindings. BLOCK and TAGBODY forms which don't serve as non-local target are not affected by this reasoning; they fall under case (2). CATCH can never be considered taking only local transfers of control as it However, the entire application may not be binding any specials at all. The overhead involves a call at the start of each block which contains non-local-transfer targets to snapshot the special bindings state and to restore it *in case of a non-local transfer*. The calls involved are markSpecialBindings() and restoreSpecialBindings(), two very short and - I think - efficient functions. So, there's a small drawback in applications which don't use special bindings much. However, the potential gain in applications which do use them is quite relevant: the example I gave before shows that a relatively simple function binding the same special variable three times will cause 3 separate specials-unwinding-exception blocks, each rethrowing the exception to the next level. Thinking about it a bit more, I realize that no additional exception blocks have to be registered. Just only the catch/restore of specials state. Looking at it this way, I propose that we stop restoring specials in finally blocks, moving part of the responsibility of having correct specials bindings to BLOCK, CATCH, TAGBODY and UNWIND-PROTECT. My change should affect both the interpreter and the compiler, to get the full benefit of efficiency. However, I thought I'd implement the LET/LET* binding optimization in the compiler first, just to see how much we benefit from this change in Maxima. Comments? Bye, Erik.

1 0

[armedbear-devel] Increased FASL size support -- was: [armedbear-cvs] r13013 - trunk/abcl/src/org/armedbear/lisp
by Erik Huelsmann 09 Nov '10

09 Nov '10

There were some reports with FASLs becoming too large -- ie generating more .cls files than can be loaded in our loader method "<fasl-name>_0" because the code overflows the 64kB method size boundary. The change below drastically reduces the size of the generated loader method, reducing the chances that people run into an issue. Of course, that can't be the final state, but it seems like a good option, as long as we don't implement the means to split this method. Implementing means to split specifically this method however, doesn't seem good enough: all of our methods have the potential of overrunning the 64kB boundary and it would be nice if we could implement general method-splitting logic instead, to be applied when methods overrun their 64kB boundary size. This commit is in our upcoming release, aleviating (but not solving) the Big FASLs problem. Bye, Erik. On Mon, Nov 8, 2010 at 11:34 PM, Erik Huelsmann <ehuelsmann(a)common-lisp.net> wrote: > Author: ehuelsmann > Date: Mon Nov 8 17:34:18 2010 > New Revision: 13013 > > Log: > Reduce the impact of the fasl-loader "_0" class's maximum size of 64kB: > this change reduces the size of pprint_0.cls from 23001 to 19380 bytes, > a saving of 18.64%, which is probably much more when counting the size > of the execute() method alone. > > Modified: > trunk/abcl/src/org/armedbear/lisp/compile-file.lisp > > Modified: trunk/abcl/src/org/armedbear/lisp/compile-file.lisp > ============================================================================== > --- trunk/abcl/src/org/armedbear/lisp/compile-file.lisp (original) > +++ trunk/abcl/src/org/armedbear/lisp/compile-file.lisp Mon Nov 8 17:34:18 2010 > @@ -679,7 +679,14 @@ > (defun generate-loader-function () > (let* ((basename (base-classname)) > (expr `(lambda (fasl-loader fn-index) > - (identity fasl-loader) ;;to avoid unused arg > + (declare (type (integer 0 256000) fn-index)) > + (identity fasl-loader) ;;to avoid unused arg > + (jvm::with-inline-code () > + (jvm::emit 'jvm::aload 1) > + (jvm::emit-invokevirtual jvm::+lisp-object+ "javaInstance" > + nil jvm::+java-object+) > + (jvm::emit-checkcast +fasl-classloader+) > + (jvm::emit 'jvm::iload 2)) > (ncase fn-index 0 ,(1- *class-number*) > ,@(loop > :for i :from 1 :to *class-number* > @@ -687,20 +694,14 @@ > (let* ((class (%format nil "org/armedbear/lisp/~A_~A" > basename i)) > (class-name (jvm::make-class-name class))) > - `(,(1- i) > - (jvm::with-inline-code () > - (jvm::emit 'jvm::aload 1) > - (jvm::emit-invokevirtual jvm::+lisp-object+ "javaInstance" > - nil jvm::+java-object+) > - (jvm::emit-checkcast +fasl-classloader+) > - (jvm::emit 'jvm::dup) > - (jvm::emit-push-constant-int ,(1- i)) > - (jvm::emit-new ,class-name) > - (jvm::emit 'jvm::dup) > - (jvm::emit-invokespecial-init ,class-name '()) > - (jvm::emit-invokevirtual +fasl-classloader+ > + `(,(1- i) > + (jvm::with-inline-code () > + (jvm::emit-new ,class-name) > + (jvm::emit 'jvm::dup) > + (jvm::emit-invokespecial-init ,class-name '()) > + (jvm::emit-invokevirtual +fasl-classloader+ > "putFunction" > - (list :int jvm::+lisp-object+) jvm::+lisp-object+) > + (list :int jvm::+lisp-object+) jvm::+lisp-object+) > (jvm::emit 'jvm::pop)) > t)))))) > (classname (fasl-loader-classname)) > > _______________________________________________ > armedbear-cvs mailing list > armedbear-cvs(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-cvs >

1 0

[armedbear-devel] Reading a UTF-8 lisp source fails.
by pjb＠informatimago.com 07 Nov '10

07 Nov '10

My sources contain some non US-ASCII characters, so they're all encoded in UTF-8. However, loading them fails: [pjb@kuiper :0.0 lisp]$ LC_CTYPE=en_US.UTF-8 abcl Armed Bear Common Lisp 0.20.0 Java 1.6.0_22 Sun Microsystems Inc. Java HotSpot(TM) Server VM Low-level initialization completed in 0.266 seconds. Startup completed in 0.726 seconds. Type ":help" for a list of available commands. CL-USER(1): (load"test-abcl-unicode.lisp") Error loading /home/pjb/src/git/public/lisp/test-abcl-unicode.lisp at line 3 (offset 54) #<THREAD "interpreter" {132021A}>: Debugger invoked on condition of type STREAM-ERROR Input value 0xffc2 is malformed while recoding with charset US-ASCII Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2): (quit) [pjb@kuiper :0.0 lisp]$ What should I do to let abcl read them as UTF-8 and not try to convert them into US-ASCII? The test file is: ------------------------------------------------------------------------ ;;;; -*- mode:lisp;coding:utf-8 -*- (defvar *test* "45°") ;;;; THE END ;;;; ------------------------------------------------------------------------ -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}.

2 1