Hi all,
Although I wanted to work on ticket #201 (&whole broken in DEFINE-METHOD-COMBINATION), I found a regression (ticket #227) which I believe is strongly correlated with how our process of auto loading works (ticket #226).
So, I've done a bit of work, some of which already committed, to extract the symbols of toplevel defined functions/methods and macros from our bootstrap build. I'm now working to use that list to create a more complete "autoloads.lisp" file.
During that work I found at least one cause of instability regarding our autoloading and the definition of generic functions: we were resolving (autoloading) symbols with an autoloader in the function slot when defining a generic function on them. Usually however, the reason to be defining a generic function on a symbol with an autoloader in the function slot will be the fact that autoloading has been triggered. The only reason this worked in the past is that our list of autoloaded symbols was so small we haven't run into this situation. Should that ever have happened, that'd have been very hard to find though.
Another thing I found is that MACROEXPAND-1 (and by extension of its use in PRECOMPILE, the precompilation) try to resolve symbols with autoloaders on them. That's a problem as well, because we have forward-referenced functions: e.g. we forward-reference SLOT-VALUE-USING-CLASS from SLOT-VALUE. When clos.lisp is being loaded and SLOT-VALUE is being defined using DEFUN, it's being "precompiled": translated to an efficient execution form. However, that process finds a reference to SLOT-VALUE-USING-CLASS, which it will autoload. This recursively triggers loading of clos.lisp.
Do we still want to do precompilation? Yes, I think so, because on my machine usual build times are ~ 6 minutes for a full build; this time goes up to ~16 minutes without precompilation. So, the process does what it's designed to do well.
The answer then - I think - is to stop precompiling during function definition and delay that until the first actual invocation.
---- breaking news:
While writing the above, I came up with another option, which is to make MACROEXPAND-1 only resolve symbols which are macro functions. That worked, or at least, that's what it looks like. I had a successful build and will run the compiled tests in a minute.
As things stand, we'd be able to start generating most of our autoloads.lisp file.
[even later: seems the test suite is completing now. I'm having a PSETF regression, I think, but no idea why yet.] -----
So, I think I'll work out the remainder of the change and make sure it gets committed. After that, I'll have a look at the PSETF failure - which is hopefully fall-out from another change or at least has a workaround in a different way: having automated autoloaders seems a valuable asset.
Bye,
Erik.