Since I'm waiting for some clients to make appropriate changes
before I may release ASDF (notably FSet and GBBopen),
I'm considering making a few additional backward-incompatible changes
in interfaces that I believe are not otherwise used
(after grepping for code in quicklisp).
The crux of the change is that I would like to move to a model where
operations are a place-holder with zero content besides the class name
— or failing that, a model where the timestamps are associated to
a specific instance of an operation, not a class name,
at which point memoization of "equivalent" operations
is necessary for #'equal hash-tables to contain timestamps.
This is not just a matter of taste, but also of correctness:
if the semantics of two instances of an operation is actually different,
then considering one is done when the other was done is incorrect
— yet that's what ASDF currently does.
I recently added a PLAN object for the purpose of traversals, which
made it possible for ASDF and POIU to share the TRAVERSE algorithm.
(Incidentally, it also makes ASDF look more like XCVB or LIL.)
I'm moving :force and :force-not to this PLAN object,
removing them (and :verbose) from OPERATION,
removing the whole OPERATION-ANCESTOR crud
(which was only a dirty way of having the ancestor operation object
behave as a kind-of backassward plan object) and
having a global hash of *OPERATIONS* that are EQUAL
iff created with the same initargs.
Now for issues:
* swank.asd relies on OPERATION-FORCED for some obscure RELOAD feature.
I think this is a mistake. I'll discuss that with the slime developers.
Meanwhile, I'll keep it aroud through a stub for backward-compatibility.
* GBBopen (again!) uses operation initargs to determine
the name of an application to be built. I think that's a mistake.
This information should be in the component.
I'll discuss the matter with the GBBopen developers.
Meanwhile, I will keep the initargs around for backward-compatibility.
* the :on-failure and :on-warnings flags to operation
don't actually work well with the new model,
and had quite fishy semantics in the old one.
Grepping through quicklisp-distributed code,
it looks like no one is actually *using* it,
though it's been cargo-culted in a few places.
I will remove it, just keeping a few stubs for backward-compatibility.
In practice, the only thing that works is to either
bind *compile-file-on-warning-behaviour*
and/or *compile-file-on-failure-behaviour* around the build, or
equivalently pass :on-warning and/or :on-failure arguments to operate.
The internal ASDF machinery that used to make that happen
is horribly complex (what with make-sub-operation,
now called find-operation, to pass along the flags
from each operation to the next in the traversal),
and does no more than if operate just recognized those keywords
to bind those variables — which is my proposed interim solution
(the defaulting based on a dynamic variable otherwise breaks my model
of operations being equal if created from the same keyword arguments).
I don't like those variables, especially since
they mean subtly different things on different implementations
(as usual CLISP being an outlier), but
I'll have to think on how to best eliminate or replace them.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
To surrender to ignorance and call it God has always been premature,
and it remains premature today. — Isaac Asimov
OK, so 2.26.45 is my new release candidate for ASDF 2.27.
Please grab it and test it.
Since last release candidate 2.26.24,
* lots of work was done towards backwards compatibility,
fixing bugs introduced or uncovered
during the previous fixing of essential conceptual bugs
through massive refactoring of internals.
* tests were added for asdf-bundle, which now also runs on XCL,
but not on ECL with bytecode compiler.
* tests were ran on all platforms with a command line on Linux x86-64
and even on LispWorks Personal Edition and Genera that don't,
and they all pass modulo a few implementation bugs that were #-'ed out.
* upgrade tests were also made to pass everywhere that's meaningful
* consequent minor fixes to warnings or corner case errors were issued for
Allegro, CLISP, ECL, ECL with bytecode compiler, SBCL, XCL.
* There was a fix for LispWorks Personal Edition.
You need to update ironclad and nibbles to run with the new ASDF
because of the dropped if-component-dep-fails feature,
but all other software should continue to run unmodified.
Since release 2.26, the main changes are as follows:
* ASDF-BUNDLE was merged into ASDF.
This notably means fewer headaches for users of ECL;
it also means a new load-fasl-op feature for all users of a modern CL,
making it easier to deliver software as a single fasl.
All remnants of its predecessor asdf-ecl were removed.
* *decades-old* bugs of notable significance were fixed:
(a) changes in dependencies now trigger a rebuild across systems, and
https://bugs.launchpad.net/asdf/+bug/479522
(b) timestamps are now correctly propagated
https://bugs.launchpad.net/asdf/+bug/1087609
In olden days, some have argued for not rebuilding systems that :depend-on
a modified one as a "feature". It really isn't. On the other hand,
if they want to explicitly prevent a rebuild, the actual feature :force-not
is there to help them (released as part of 2.21 in April 2012).
I will write about those bugs.
* PREPARE-OP was introduced to fix a conceptual bug in the ASDF object model.
It corresponds to "loading the dependencies of a component and its parents"
and is explicitly depends-on'ed before to load or compile the component,
instead of implicitly depending on it via magic in the TRAVERSE algorithm,
which magic proved insufficient in cases revealed by the above bugs.
It propagates upward in the component hierarchy, rather than downward,
like the TRAVERSE algorithm used to systematically propagate dependencies.
* TRAVERSE was gutted out and factored into reusable higher-order functions,
which not only fixes the above conceptual bug, but makes for semantics
that are simpler to implement, possible to understand, easier to extend,
and less limited in expressivity. TRAVERSE doesn't automatically propagate
operations downward the component hierarchy anymore;
propagation is now implemented through methods on COMPONENT-DEPENDS-ON.
* COMPONENT-DEPENDS-ON is now more powerful: it can express dependencies
on arbitrary operation objects (i.e. operations of the same class
can have different options) acting on arbitrary component objects
(i.e. not necessarily siblings of the current component).
The ASDF object model can now express arbitrary build graphs, with
special magic no more for children components vs other dependencies.
* Internals have been refactored, and some sorry features were excised.
The semantics of OPERATION-DONE-P is simplified and now well-specified.
The dubious DOFIRST mechanism is gone.
FIND-COMPONENT will pass component objects through, and
a corresponding FIND-OPERATION replaces MAKE-SUB-OPERATION.
Several internal accessors were renamed.
* IF-FEATURE is a new attribute of components that accepts an arbitrary
feature expression such as (:and :sbcl (:or :x86 :x86-64)),
which when defined is a precondition to the component being enabled;
when the expression is false, the component is disabled,
and dependencies to it are removed.
It replaces the misguided :if-component-dep-fails attribute of modules
and the accompanying :feature feature, which were dropped.
They were not very expressive, clunky to use, and relied on baking
conceptually dubious non-local behavior in the old TRAVERSE algorithm,
which belied the object model and defeated other features.
Unhappily, this breaks backwards compatibility for the few who used it,
namely sb-grovel, nibbles and ironclad.
A minimal compatibility layer allows ASDF 2.27 or later to load
the SB-GROVEL of old versions of SBCL, but it won't suffice to load
an old version of nibbles or ironclad; please upgrade them
a recent version that was updated to support ASDF 2.27.
* ASDF2.27 was added to the *features* so you can detect the presence of
this massively updated ASDF and its new features with #+asdf2.27.
This is *not* ASDF3, for we painfully maintained backwards compatibility.
* Portability updates were done for
ABCL, CLISP, CMUCL, ECL, LispWorks Personal Edition, MKCL, XCL.
* Documentation received some small updates.
* Tests were updated, notably including support for asdf-bundle.
They were successfully run on ccl clisp sbcl ecl ecl_bytecodes
cmucl abcl scl allegro lispworks allegromodern xcl.
Manual tests were run on lispworks-personal-edition genera.
Untested were cormancl mkcl rmcl.
* POIU was updated. This extension is distributed separately from ASDF.
It is now much simplified and made more correct by construction by
reusing the new TRAVERSE infrastructure of ASDF and now being able to
assume the ASDF object model is a complete description of dependencies.
On SBCL it will handle deferred warnings from background compilations.
On platforms that cannot fork it will gracefully fall back
to serial compilation.
* SLIME support for ASDF was significantly enhanced.
It is distributed separately from ASDF, with SLIME.
See latest checkin from 2012-12-26.
We recommend that in your ~/.swank.lisp you should:
(in-package :swank)
(defparameter *upgrade-asdf-p* t)
(pushnew 'try-compile-file-with-asdf *compile-file-for-emacs-hook*)
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
He wa'n't no common dog, he wa'n't no mongrel; he was a composite.
A composite dog is a dog that is made up of all the valuable qualities
that's in the dog breed — kind of a syndicate; and a mongrel is made up
of all riffraff that's left over. — Mark Twain
Another conceptual bug in ASDF.
Checking the plans generated for load-fasl-op, I find that with my new
ASDF 2.26.x (currently x=45), it systematically loads all the
individual .fasl's before it loads the .system.fasl, defeating the
purpose.
This was not the case before: in 2.26.7 and earlier, the (separately
distributed before 2.26.7) asdf-bundle has load-fasl-op depend-on
fasl-op which depends-on compile-op, which doesn't depend-on load-op,
but do-first's on it — so that the dependency is only considered but
if the immediate timestamp of the fasl is older than the timestamp of
its lisp source. If all fasl's satisfy this local timestamp check,
nothing is recompiled, none of the do-first is triggered, and the
load-op's are not attempted. But the old design also fails to
propagate timestamp information from the other files that are
depended-on, let alone do-firsted. Therefore, there is some notion in
the old do-first that must be preserved in the new ASDF, even though
the previous design was conceptually flawed overall.
In the end, what must be preserved is that some dependencies need to
have been performed in the current image, whereas other dependencies
only need to have been performed in some previous (or current) image.
This means a dependency can be walked over twice: once in without
"image-necessity", and a second time with it. Without image-necessity,
only the timestamp matters; with image-necessity, whether it was
performed in current image matters, too. Presumably, a node with
output-files clears the image-necessity flag before checking
timestamps, but re-traverses its dependencies with the flag if the
timestamp check fails. A node with no output-files on the other hand
is done for in-image effect, and heeds and propagates the
image-necessity flag to its subnodes. I believe that it's possible to
continue the current strategy of traversal without an explicit graph,
though nodes may now be traversed twice: once without image-necessity,
once with. Sigh.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Success is getting what you want. Happiness is wanting what you get.
— Dale Carnegie
