November 2009 - armedbear-devel - mailman3.common-lisp.net

Re: [armedbear-devel] 0.18 and beyond
by logicmoo＠gmail.com 06 Nov '09

06 Nov '09

Dissregared the last email there was a Hypelink that I misspelled as well as rewrote my last post to make clearer ----- Original Message ----- From: "Ville Voutilainen" <ville.voutilainen(a)gmail.com> To: <dmiles(a)users.sourceforge.net> Cc: "Armed Bear" <armedbear-devel(a)common-lisp.net> Sent: Thursday, November 05, 2009 9:57 PM Subject: Re: [armedbear-devel] 0.18 and beyond > 2009/11/6 <logicmoo(a)gmail.com>: >>>> Other java lisps define each Primitive execute as a single static function >>>> in a trampolines file. >>>> The Primitive itself just calls the static function in the trampolines ( >>>This is a useful technique to speed up compiled code. It could also >>>speed up startup times as long as you don't invoke such primitives >> Indeed > > The translation looks like there's no longer a Primitive that extends > LispObject. No longer a "requirement" for the compiler to *have to* use a Primitive > Do we generate wrappers on the fly? Correct As you foresee, for cases like #'APPLY/#'FUNCALL etc and from the interpreter it is done on the fly. Previous code that read: declareFunction(myName,"primitive_CHAR_LESS_THAN_execute_1_0_false","CHAR<",1,0,false); declareFunction(myName,"primitive_CHAR_LESS_THAN_execute_2_0_false","CHAR<",2,0,false); declareFunction(myName,"primitive_CHAR_LESS_THAN_execute_0_0_true","CHAR<",0,0,true); Does actually create 5 objects: One Symbol: Symbol s = intern("CHAR<",pkg); One Primtive subclass called a FunctorIndexer FunctorIndexer p = new FunctorIndexer(s); One Primtive is placed inside the symbol-function: s.setSymbolFunction(p); Three Functors: p.Add( new ForwardFunctor(s,myName,"primitive_CHAR_LESS_THAN_execute_1_0_false","CHAR<",1,0,false)); p.Add ( new ForwardFunctor(s,myName,"primitive_CHAR_LESS_THAN_execute_2_0_false","CHAR<",2,0,false)); p.Add( new ForwardFunctor(s,myName,"primitive_CHAR_LESS_THAN_execute_0_0_true","CHAR<",0,0,true)); This will later be used to index the call sigs to the best ForwardFunctor. So far we have made 5 objects: s,p,f0,f1,f2 ------------------------------------------------------------------------------------------ On first use: lets say: (funcall 'char< #\a #\b #\c) thats 3 args ins .. the best match is f2 - This transistion is handled in here: http://larkc.svn.sourceforge.net/viewvc/larkc/trunk/platform/src/com/cyc/to… It creates one class that implments processArgs(LispObject r0,LispObject r1,LispObject r2) And loads the arity3Cache of the FunctorIndexer which physically calls the static function: return primitive_CHAR_LESS_THAN_execute_0_0_true(new LispObject[]{r0,r1,r2}); ----------------------- So far we have made 6 objects ----------------------- Note though this machinery is only used in FUNCALL and interpreted code When hitting the functor generator .. it should be considered "worse case" But serves a better way case than hitting the reflection on line 62 of http://larkc.svn.sourceforge.net/viewvc/larkc/trunk/platform/src/com/cyc/to… (But this 2ndary wrapper technique still isn't that bad. ) > How do you then pass the primitive as one? Like passing EQ/EQUAL/EQUALP > to FIND/POSITION family of functions? You find-or-create the Symbol like findSymbol("EQ",pgk).getSymbolFunction().execue(a0,a1,etc); like before. ".getSymbolFunction()" is where the new magic takes place > Or user-defined functions that take > functions, and you don't know the type and must box it (aka pass it as > a LispObject)? Currently all execute(..,.,.,.) signatures take LispObject as before. However with not much more work it is possible to make a cache of unboxed versions like... public final static LispObject primitive_CHAR_LESS_THAN_execute_2_0_false_nobox(char req0, char req1) { return req0 < req1? T : NIL; } Registered the same way to the interpreter and compiler: declareFunction(myName,"primitive_CHAR_LESS_THAN_execute_2_0_false_unboxed","CHAR<",2,0,false); Actually if you want, is even simplier to not go thru the explicit "registration process" but using Annotations @InlineCompiler(symbolname="CHAR<", package="COMMON-LISP"); public final static LispObject primitive_CHAR_LESS_THAN_execute_2_0_false_nobox(char req0, char req1) { return req0 < req1? T : NIL; }

1 0

[armedbear-devel] symbol-macrolet bug
by szergling 05 Nov '09

05 Nov '09

Hi everyone, I've been quite stoked to watch the surge of activity on armedbear in the last few months. Keep up the good work guys! My reason for this email is: If I type the following at the REPL, or load from a file (that's how I discovered this), I get an error in abcl-0.16.0. Regardless of mode (REPL. load file, with or without eval-when), I always get an error. It's that load-time-value causing trouble, I believe... (defun test () (symbol-macrolet ((ii (load-time-value 45))) ii)) (test) #+nil ;; uncomment for alternative test (eval-when (:load-toplevel :execute) (test)) The symbol-macro seems to have disappeared for some reason... Debugger invoked on condition of type UNBOUND-VARIABLE: The variable II is unbound. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(78): Apologies if this is a well known bug. Thank you!

2 2

[armedbear-devel] ABCL basics
by dusan lukic 05 Nov '09

05 Nov '09

Hi all, with your help, I managed to create abcl.jar file, and now I need some basic knowledge of using it. I would like to know how to compile Lisp code (in a form of String or text file) and call it from Java code. I would also like to get some basic idea of which classes are defined in the abcl.jar file, and what is their purpose and what can I do with them. Furthermore, I would like to obtain some abcl help (it can be online, too), which contain descriptions of abcl classes and their methods, fields, stuff like that. Can anyone point me to some documentation or give some guidelines? Thaks.

2 1

[armedbear-devel] Fwd: [j-devel] Improving startup time: sanity check
by Erik Huelsmann 03 Nov '09

03 Nov '09

forwarding ---------- Forwarded message ---------- From: Alessio Stalla <alessiostalla(a)gmail.com> Date: Wed, Oct 28, 2009 at 11:20 PM Subject: Re: [j-devel] Improving startup time: sanity check To: Erik Huelsmann <ehuels(a)gmail.com> Cc: armedbear-j-devel(a)lists.sourceforge.net, Alex Muscar <muscar(a)gmail.com> On Wed, Oct 28, 2009 at 9:15 PM, Erik Huelsmann <ehuels(a)gmail.com> wrote: > Last weekend, we experimented with better autoloading. It turned out > to strip roughly .4 seconds from a cold startup time of 1.7s, making > it a 25% improvement. > > However, the reason we started out with the startup time improvements > in the first place was the ABCL startup time on Google App Engine. It > turns out that our CPU usage during startup hasn't really decreased > much (as per their benchmark indicator - they can't really give an > actual figure). > > So, I asked for advice on #appengine (on freenode). Their reaction was > "we can't imagine the startup time being related to the size of the > JAR" even though Peter Graves calculated a 34% ratio between ABCL and > Clojure jar sizes and a 35% ratio between startup times - that looks > like a linear match. Their reaction continued "you're probably just > doing too much work during the init() phase." > > The init() phase is where the ABCL environment gets loaded and all > function objects get created. > > Let's assume for a second they're right. In that case we must assume > it's not I/O holding us up: it's the work the CPU must do to get us up > and running. If that's true, profiling the application should tell us > something about the bottlenecks we're running into. I happen to have > done quite a number of such profiles in the course of last week. The > conclusion which stands out is that ABCL - during the startup process > - spends ~ 40% of its time finding class constructors: the main > component of creating function objects. > > This brought me to the conclusion that our startup process could be > much faster, if we decided to delay function object creation until the > function is actually used: we would eliminate the need to construct > function objects until they're used instead of creating them when > their siblings are requested to be loaded. > > The idea is to create another Autoload derivative which will be > "installed" in the appropriate places which, when invoked, loads the > actual class from the byte array. I'm hoping this will cause a more > equally spread "initialization load". The performance hit will only be > the first call to the function: after it has been converted from the > byte array, the autoload object will remove itself from the function > call chain. > > So, how about it? Comments most welcome! I have mixed feelings about the idea. I think it's clever; but I also think we (I, at least) need more data to know if it will be actually beneficial. If the goal is speeding up startup time in a context like AppEngine - where not only Lisp, but the whole user application will be loaded from scratch from time to time - then it is critical to know how many Lisp functions a generic application uses on average (both directly and indirectly). If it turns up that, say, 50% of Lisp is commonly used, then no matter how clever an autoloading scheme you implement, you'll cut loading times only by roughly 50% at best. If getting constructors through reflection is really the bottleneck, and if we determine that using new instead of reflection is significantly faster (from a quick test of mine, it seems it *really* is [1]), then it might be sensible to avoid reflection altogether and devise another scheme. For example, the compiler-generated class X could contain in its static initialization block the equivalent of something like Lisp.someThreadLocal.set(new X()) and loadCompiledFunction or what it is could just fetch the instance from the threadlocal; not very elegant, but if it speeds things up... Alessio [1] this is the astounding result on a couple of runs on 50000 iterations (test files attached): REFLECTION: 16262373155 NEW: 84267527 % SLOWER: 19298 REFLECTION: 15917190176 NEW: 103681915 % SLOWER: 15351 REFLECTION: 15838714133 NEW: 77235481 % SLOWER: 20507 (times in ns) i.e. reflection as we use it is roughly 150-200 times slower than new and that's on a very simple class with no superclasses and a single constructor! The test might be wrong as I wrote it quickly and it's quite tricky. It uses the very same classloader of abcl, though (copy-pasted).

3 3

[armedbear-devel] Focusing 0.18 efforts
by Erik Huelsmann 03 Nov '09

03 Nov '09

Today, I branched for 0.17 (to be released soon), meaning that trunk has become free for 0.18 development. Personally, I found there are a number of items open any one of which could become part of our 0.18 effort. These are the points I identified: * Investigate and improve startup performance * CLOS conformance improvements (including define-method-combination support?) * Cleanup of Load.java - which would allow loading compressed fasls off streams * Gray stream compatibility finalization [we're 99% compatible, but FUNDAMENTAL-STREAM should be recognized as a subtype of STREAM] * Some random other stuff that comes along Do you have your own, or opinions on the points I mentioned above? Let's hear it! Bye, Erik.

2 3

[armedbear-devel] One more change for 0.17: FASL format update
by Erik Huelsmann 03 Nov '09

03 Nov '09

During the coming into existence of 0.17, there was an API change with respect to binding specials. This api change allows quick local access to the binding generated. However, the function for which the api signature is changed, is part of our generated .cls files (FASLs). Hence, I conclude we need a FASL version increment before 0.17 - which hasn't happened yet. Regards, Erik.

1 0

[armedbear-devel] Fw: [jvm-l] Re: Some Array vs GETFILED access times
by logicmoo＠gmail.com 02 Nov '09

02 Nov '09

----- Original Message ----- From: "Charles Oliver Nutter" <headius(a)headius.com> To: <jvm-languages(a)googlegroups.com> Sent: Sunday, November 01, 2009 8:47 PM Subject: [jvm-l] Re: Some Array vs GETFILED access times None of your results are too surprising to me. In JRuby, we moved several structures from encapsulating an array to using differently-sized "slotted" objects, with performance registering much better as a result. Some of that was due to the lack of boundschecks on the arrays, but another portion was probably due to the reduced size of an object + reference versus object + reference + array. Some comments inline below. On Sun, Nov 1, 2009 at 7:48 PM, <logicmoo(a)gmail.com> wrote: > public class ArrayVsClass { > > public static void main(String[] args) { > long lVectorStartTime = System.currentTimeMillis(); > int iterations = Integer.MAX_VALUE; > while (iterations-- > 0) { > // int iterations2 = 10; > //while (iterations2-- > 0) > { > //testArray(); // ARRAY > // vs > //testGETFIELD(); // GETFIELD > // vs > testIBean(); // INVOKE INTERFACE > // vs > //testBean(); // INVOKE VIRTUAL > // vs > //testABean(); // INVOKE VIRTUAL POSSIBLY THROW > // vs > //testSlots(); // INVOKE FOR AVALUE > } > } > long lVectorRunTime = System.currentTimeMillis() - lVectorStartTime; > System.out.println("Bench time: " + lVectorRunTime); > > } Because of optimization effects, you should try running them all together in the same benchmark in varying orders. Short benchmarks like these can skew actual results because only a small subset of the full code needs to be considered for optimization. > // SLOTS time: 33157,33250,33156 > public static void testSlots() { > ClassWithSlots oneSlot = new ClassWithSlots(6); ... > // Array time: 18438,18437,18422 > public static void testArray() { > final long[] accessArray = new long[] { 6 }; Not too surprising; you're paying the cost of the array plus the cost of the virtual invocation. So even after inlining, you've got something like boundscheck + deref + virtual call. > // GETFIELD time: 14688,14531,14453 > public static void testGETFIELD() { > ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); ... > // INVOKE VIRTUAL time: 14750,14594,14719 > public static void testBean() { > ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); This is exactly the pattern we use in JRuby for heap-based scopes. We have from ZeroVarDynamicScope up to FourVarDynamicScope and then it falls over into an array-based version. Because we can statically tell how many variable slots we'll need in most Ruby scopes, this ended up being a big perf improvement for us. > // INVOKE INTERFACE time: 14469,14610,14859 > public static void testIBean() { > IBeanWithOneSlot oneSlot = new ClassWithOneSlot(6); > > int iterations = Integer.MAX_VALUE; > long result = 0; > while (iterations-- > 0) { > result += oneSlot.getValue(); > } > } invokeinterface ends up as fast as invokevirtual once it's been inlined, so this is no surprise. > // INVOKE VIRTUAL POSSIBLY THROW time: 14641,14594,14547 > public static void testABean() { > AClassWithOneSlot oneSlot = new ClassWithOneSlot(6); Exception-handling paths not followed do not impact performance, so this is also not surprising. Try having one out of the N invocations trigger the exception and watch the perf change drastically from then on. - Charlie --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JVM Languages" group. To post to this group, send email to jvm-languages(a)googlegroups.com To unsubscribe from this group, send email to jvm-languages+unsubscribe(a)googlegroups.com For more options, visit this group at http://groups.google.com/group/jvm-languages?hl=en -~----------~----~----~----~------~----~------~--~---

1 0

[armedbear-devel] Some Array vs GETFILED access times
by logicmoo＠gmail.com 02 Nov '09

02 Nov '09

ABCL, For implementing MOP-like objects these times are something to consider. What I draw from it is when not messing with arrays (array bounds checking). Every case is better than. JVM-L, Do these tests with times make sense? package ArrayTests; public class ArrayVsClass { public static void main(String[] args) { long lVectorStartTime = System.currentTimeMillis(); int iterations = Integer.MAX_VALUE; while (iterations-- > 0) { // int iterations2 = 10; //while (iterations2-- > 0) { //testArray(); // ARRAY // vs //testGETFIELD(); // GETFIELD // vs testIBean(); // INVOKE INTERFACE // vs //testBean(); // INVOKE VIRTUAL // vs //testABean(); // INVOKE VIRTUAL POSSIBLY THROW // vs //testSlots(); // INVOKE FOR AVALUE } } long lVectorRunTime = System.currentTimeMillis() - lVectorStartTime; System.out.println("Bench time: " + lVectorRunTime); } // SLOTS time: 33157,33250,33156 public static void testSlots() { ClassWithSlots oneSlot = new ClassWithSlots(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } // Array time: 18438,18437,18422 public static void testArray() { final long[] accessArray = new long[] { 6 }; int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += accessArray[0]; } } // GETFIELD time: 14688,14531,14453 public static void testGETFIELD() { ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.slot; } } // INVOKE VIRTUAL time: 14750,14594,14719 public static void testBean() { ClassWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } // INVOKE INTERFACE time: 14469,14610,14859 public static void testIBean() { IBeanWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } // INVOKE VIRTUAL POSSIBLY THROW time: 14641,14594,14547 public static void testABean() { AClassWithOneSlot oneSlot = new ClassWithOneSlot(6); int iterations = Integer.MAX_VALUE; long result = 0; while (iterations-- > 0) { result += oneSlot.getValue(); } } static interface IBeanWithOneSlot { public long getValue(); } static class ClassWithOneSlot extends AClassWithOneSlot implements IBeanWithOneSlot { final public long slot; ClassWithOneSlot(long s) { slot = s; } @Override final public long getValue() { return slot; } } static class ClassWithSlots { final public long[] slots = new long[1]; ClassWithSlots(long s) { slots[0] = s; } final public long getValue() { return slots[0]; } } static abstract class AClassWithOneSlot implements IBeanWithOneSlot { @Override public long getValue() { throw new NullPointerException(); } } }

1 0