- armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] Autoloader progress and experiences
by Erik Huelsmann 31 Jul '12

31 Jul '12

Hi all, Although I wanted to work on ticket #201 (&whole broken in DEFINE-METHOD-COMBINATION), I found a regression (ticket #227) which I believe is strongly correlated with how our process of auto loading works (ticket #226). So, I've done a bit of work, some of which already committed, to extract the symbols of toplevel defined functions/methods and macros from our bootstrap build. I'm now working to use that list to create a more complete "autoloads.lisp" file. During that work I found at least one cause of instability regarding our autoloading and the definition of generic functions: we were resolving (autoloading) symbols with an autoloader in the function slot when defining a generic function on them. Usually however, the reason to be defining a generic function on a symbol with an autoloader in the function slot will be the fact that autoloading has been triggered. The only reason this worked in the past is that our list of autoloaded symbols was so small we haven't run into this situation. Should that ever have happened, that'd have been very hard to find though. Another thing I found is that MACROEXPAND-1 (and by extension of its use in PRECOMPILE, the precompilation) try to resolve symbols with autoloaders on them. That's a problem as well, because we have forward-referenced functions: e.g. we forward-reference SLOT-VALUE-USING-CLASS from SLOT-VALUE. When clos.lisp is being loaded and SLOT-VALUE is being defined using DEFUN, it's being "precompiled": translated to an efficient execution form. However, that process finds a reference to SLOT-VALUE-USING-CLASS, which it will autoload. This recursively triggers loading of clos.lisp. Do we still want to do precompilation? Yes, I think so, because on my machine usual build times are ~ 6 minutes for a full build; this time goes up to ~16 minutes without precompilation. So, the process does what it's designed to do well. The answer then - I think - is to stop precompiling during function definition and delay that until the first actual invocation. ---- breaking news: While writing the above, I came up with another option, which is to make MACROEXPAND-1 only resolve symbols which are macro functions. That worked, or at least, that's what it looks like. I had a successful build and will run the compiled tests in a minute. As things stand, we'd be able to start generating most of our autoloads.lisp file. [even later: seems the test suite is completing now. I'm having a PSETF regression, I think, but no idea why yet.] ----- So, I think I'll work out the remainder of the change and make sure it gets committed. After that, I'll have a look at the PSETF failure - which is hopefully fall-out from another change or at least has a workaround in a different way: having automated autoloaders seems a valuable asset. Bye, Erik.

1 0

[armedbear-devel] Autoloading && method definitions
by Erik Huelsmann 30 Jul '12

30 Jul '12

Hi all, Today I've been playing around with ways to automatically extract symbols from our build to automate (most) of the autoloads.lisp maintenance. So, I've created a small adjustment to COMPILE-FILE which saves the top-level DEFUNs, DEFMETHODs and DEFGENERICs which have been compiled. The result is interesting even in light of today's file: analysis shows we have several generic functions for which methods have been defined across multiple files. An example: we have definitions for PRINT-OBJECT in print-object.lisp, but also in java.lisp and format.lisp. This is a problem, because the autoloader has been instructed to load "print-object.lisp" only, when finding references to the print-object generic function. My conclusion so far is that we need to be able to trigger auto-loading of more than one source file *and* that we need a way to determine in which order to load them: we'd typically want to load the file containing the DEFGENERIC before we load the one with the other method definitions. While the former isn't trivial (think threading) but doable, I'm still thinking how to achieve the latter. So, why do we want autoloading at all? Well, we want to make sure we get a short start up time and to achieve that, we don't load the parts of the system that we don't need. How much extra time it would take to load the full system? I have no idea at this moment. (But we could experiment and code to that extent would be greatly appreciated!) And why do I want to part with our current system? Well, from analysis of the DEFUN/DEFGENERIC/DEFMETHOD data (and thus excluding the DEFMACRO data) we have 2512 definitions each of which may be referenced by our users. Currently we define a subset "likely to be used" which causes autoloading to happen. However, since we regularly break that subset ourselves, I want to automate the task by collecting the symbols during the build (and allow for manual additions and deletions - which should be much more rare). Any comments, complementing experiments or proposed solutions are very welcome! Bye, Erik.

1 0

[armedbear-devel] bug in FASL loader
by Durward McDonell 29 Jul '12

29 Jul '12

Hello. I believe I have found a bug in FASL loading, with respect to *read-base* being reset. At least, I get the behavior I expect in sbcl, and not in abcl. Consider the following code: (defmacro foo-bar (s) `(sublis '((foo . bar)) ,s)) (defun foo-bar-rb () (let ((*read-base* #x10) (it (read))) (eval it))) Load this code, then execute (foo-bar-rb). It will wait for input. Type (foo-bar '(foo)). I would expect this to evaluate to (bar), but abcl gives a FASL version mismatch, where it seems that it is reading the FASL in the new base (16 instead of 10), and reports "found '56' but expected '38' in sublis". -- Durward McDonell durward.mcdonell(a)jhuapl.edu 240-228-2690 (DC) 443-778-2690 (Balt.)

4 7

[armedbear-devel] Planning for abcl-1.1.0
by Mark Evenson 29 Jul '12

29 Jul '12

Yesterday (and today), Erik and I spent time discussing and preparing for the engineering work necessary to get abcl-1.1.0 released. This release will obviously feature Rudi's hard work as a reasonably stable version of AMOP that CLOSER-MOP can be patched to reference, but we have other goals as well which I want to capture so I now transcribe a bit about our thinking, decisions, and plans going forward. Comments, objections, innovations, etc. solicited. We now have [some 34 active bugs][abcl-1.1.0] in the ABCL 1.1.0 milestone that we intend to address. We have assigned an owner to most of the bugs of either Erik or myself. This does not mean that others shouldn't be working on the bug as well if they have time to contribute to it, just that they should co-ordinate to the "owner" (a quick #abcl message like "ehu: I have stuff for #19?, do you have any patches?" would be fine to avoid duplication). Roughly, Mark got everything dealing with abcl-contrib and Pathnames, and Erik got everything dealing with the compiler. We intend to have a weekly meeting on #abcl, tentatively for set for Mondays around 2100 CET which we will review what we have gotten done, ask questions, and generally try to move the release forward. Everyone is welcome to contribute feedback, volunteer help, etc. as these will be open meetings. We plan on an abcl-1.1.1 maintenance release 30 days after abcl-1.1.0 that will fix everything that we can. [abcl-1.1.0]: http://trac.common-lisp.net/armedbear/milestone/1.1.0 An explanation of the criteria for bugs we selected: # Criteria for issues In order of importance: 0. Enable CLOSER-MOP. 1. No regressions with respect to the behavior of abcl-1.0.1. Fortunately, there only seems to be one categorized with "regression" tag (is this really the correct situation?) 2. Load a wider range of Quicklisp systems. Fix the ANSI test-grid problem if it is still active, using the source code distributed by Quicklisp as the basis for a more robust system. We might not necessarily fix the compiler in all cases, but instead implement a fallback strategy to include source forms in our FASLs for which to pass to EVAL for forms that we can't compile. This is under active consideration, as it presents the single largest unknown in terms of necessary resources to address. 3. Fix all known "ansi-conformance" problems. 4. Fix everything reported as broken by users. In the weekly meetings, we may start "jettisoning" some issues for later releases as time goes on. We estimate that given just Erik's and my time, it will take roughly three months from now to release abcl-1.1.0. If you want to make this happen faster, we welcome all helping hands. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare it to now."

1 0

[armedbear-devel] Space leak due to threads stuck in LispThread map
by Pendergrass, James A. 27 Jul '12

27 Jul '12

Hello all, I'm having some trouble with space leakage that looks to be because the static map in LispThread isn't letting go of completed JavaThreads. Basically what I'm doing on the Java side looks like: > class Foo { > public static int main(){ > manager = new Manager(); > fun = getLispFunction("processResults"); > > new Thread() { > public void run() { > LispObject res = generateSomeLisp(); > fun.execute(manager, res); > }.run(); > manager = null; > while(1); > } > } The lisp side looks like: > (defun processResults (manager res) > (let ((*manager* manager)) > (declare (special *manager*)) > (eval res))) But the manager never gets destroyed. I used the eclipse Java Memory Analysis tool to track this down to the created JavaThread being referenced as a key in the static map variable in LispThread. Any suggestions on how to get around this? The whole code base is pretty memory intensive so garbage collecting that manager is really a must (and cleaning up the thread is probably a good plan too). Thanks, -Aaron

3 6

[armedbear-devel] Saturday ABCL hacking meeting on #abcl from Austria
by Erik Huelsmann 26 Jul '12

26 Jul '12

Hi all, Mark and I are going to work together this Saturday all day from my holiday address in Austria to see if we can work on tickets (especially ANSI compliance and "needs tests"), test trunk with the latest Quicklisp libraries, and hopefully find out what's required to get to 1.1.0. We'd like to invite you into #abcl on freenode.org and get your input and participation: we'll be logged in most of the day and evening. (To those who don't know: our nicks are easye (Mark) and ehu (me, obviously)). Bye, Erik.

2 1

[armedbear-devel] asdf 2.23 released
by Faré 20 Jul '12

20 Jul '12

I am pleased to announce the release of ASDF 2.23. Since previous release 2.22, the changes are as follow: * add new :compile-check argument to compile-file* and around-compile-hook. This allows for consistency check of .fasl output before the file can possibly be seen by other builds (i.e. no race condition). This change makes the asdf-finalizers extension possible. * export more utilities for dealing with pathnames from environment. These changes are few and subtle, but nonetheless significant to those who'd use these features. As usual, developers, please test and enjoy, vendors, please update your implementation or distribution. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org There is only one basic human right, the right to do as you damn well please. And with it comes the only basic human duty, the duty to take the consequences. — P. J. O'Rourke

2 1

[armedbear-devel] Longs, Bytes, or casting in general
by Theam Yong Chew 20 Jul '12

20 Jul '12

Hi folks, I was wondering how we can conveniently pass longs, bytes into normal Java functions/constructors, for example, (jnew "java.lang.Byte" 23) ==> Java exception 'java.lang.NoSuchMethodException: Byte(java.lang.Integer)'. [Condition of type JAVA-EXCEPTION] Here only constructor available is for "byte" inputs. It's a pain to have to use this every time we use a byte, long, or similar: (jnew (jconstructor "java.lang.Byte" "byte") 23) #<java.lang.Byte 23 {3A9300BF}> Relatedly/alternatively, is there a way convenient to coerce values to a certain type, forcing 23 above (default int/Integer) to be a byte or long? Yong

3 5

[armedbear-devel] wrong FASL version with 1.0.1
by Durward McDonell 18 Jul '12

18 Jul '12

Hello. I'm using abcl-1.0.1. My code loads just fine, but when I try to run it, I get FASL version mismatch; found '38' but expected '26' in sublis I'm running this in slime. If I do something like (sublis '((foo . bar)) '(foo)) it works as expected, and THEN I can run my code and it works. Is there something cached somewhere I need to blow away, or is something else I need to do? If I look at the abcl.jar I'm using, I can see that sublis.abcl says it's FASL version 38, just like everything else in abcl.jar. Thanks. -- Durward McDonell durward.mcdonell(a)jhuapl.edu 240-228-2690 (DC) 443-778-2690 (Balt.)

2 1

[armedbear-devel] cffi, asdf question
by Durward McDonell 12 Jul '12

12 Jul '12

Hello. I noticed that there is currently some difficulty getting cffi to load properly. I've gotten it to load using Mark Evenson's patches (I think), but it still required loading something, getting an error, and loading it again, and then assuming it was loading correctly. In the interest of doing it The Right Way, I have some questions. I hope they make some sense. ASDF doesn't know about loading systems from jar files -- that seems to be the job of abcl-asdf. But abcl-asdf has to be loaded from a jar file, right? So there's a bootstrapping problem. Where is this dealt with? Or is there some other magic I haven't yet figured out to load systems from jar files? Thanks. -- Durward McDonell durward.mcdonell(a)jhuapl.edu 240-228-2690 (DC) 443-778-2690 (Balt.)

2 1