OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
Since last release candidate 2.26.24, * lots of work was done towards backwards compatibility, fixing bugs introduced or uncovered during the previous fixing of essential conceptual bugs through massive refactoring of internals. * tests were added for asdf-bundle, which now also runs on XCL, but not on ECL with bytecode compiler. * tests were ran on all platforms with a command line on Linux x86-64 and even on LispWorks Personal Edition and Genera that don't, and they all pass modulo a few implementation bugs that were #-'ed out. * upgrade tests were also made to pass everywhere that's meaningful * consequent minor fixes to warnings or corner case errors were issued for Allegro, CLISP, ECL, ECL with bytecode compiler, SBCL, XCL. * There was a fix for LispWorks Personal Edition.
You need to update ironclad and nibbles to run with the new ASDF because of the dropped if-component-dep-fails feature, but all other software should continue to run unmodified.
Since release 2.26, the main changes are as follows:
* ASDF-BUNDLE was merged into ASDF. This notably means fewer headaches for users of ECL; it also means a new load-fasl-op feature for all users of a modern CL, making it easier to deliver software as a single fasl. All remnants of its predecessor asdf-ecl were removed.
* *decades-old* bugs of notable significance were fixed: (a) changes in dependencies now trigger a rebuild across systems, and https://bugs.launchpad.net/asdf/+bug/479522 (b) timestamps are now correctly propagated https://bugs.launchpad.net/asdf/+bug/1087609 In olden days, some have argued for not rebuilding systems that :depend-on a modified one as a "feature". It really isn't. On the other hand, if they want to explicitly prevent a rebuild, the actual feature :force-not is there to help them (released as part of 2.21 in April 2012). I will write about those bugs.
* PREPARE-OP was introduced to fix a conceptual bug in the ASDF object model. It corresponds to "loading the dependencies of a component and its parents" and is explicitly depends-on'ed before to load or compile the component, instead of implicitly depending on it via magic in the TRAVERSE algorithm, which magic proved insufficient in cases revealed by the above bugs. It propagates upward in the component hierarchy, rather than downward, like the TRAVERSE algorithm used to systematically propagate dependencies.
* TRAVERSE was gutted out and factored into reusable higher-order functions, which not only fixes the above conceptual bug, but makes for semantics that are simpler to implement, possible to understand, easier to extend, and less limited in expressivity. TRAVERSE doesn't automatically propagate operations downward the component hierarchy anymore; propagation is now implemented through methods on COMPONENT-DEPENDS-ON.
* COMPONENT-DEPENDS-ON is now more powerful: it can express dependencies on arbitrary operation objects (i.e. operations of the same class can have different options) acting on arbitrary component objects (i.e. not necessarily siblings of the current component). The ASDF object model can now express arbitrary build graphs, with special magic no more for children components vs other dependencies.
* Internals have been refactored, and some sorry features were excised. The semantics of OPERATION-DONE-P is simplified and now well-specified. The dubious DOFIRST mechanism is gone. FIND-COMPONENT will pass component objects through, and a corresponding FIND-OPERATION replaces MAKE-SUB-OPERATION. Several internal accessors were renamed.
* IF-FEATURE is a new attribute of components that accepts an arbitrary feature expression such as (:and :sbcl (:or :x86 :x86-64)), which when defined is a precondition to the component being enabled; when the expression is false, the component is disabled, and dependencies to it are removed. It replaces the misguided :if-component-dep-fails attribute of modules and the accompanying :feature feature, which were dropped. They were not very expressive, clunky to use, and relied on baking conceptually dubious non-local behavior in the old TRAVERSE algorithm, which belied the object model and defeated other features. Unhappily, this breaks backwards compatibility for the few who used it, namely sb-grovel, nibbles and ironclad. A minimal compatibility layer allows ASDF 2.27 or later to load the SB-GROVEL of old versions of SBCL, but it won't suffice to load an old version of nibbles or ironclad; please upgrade them a recent version that was updated to support ASDF 2.27.
* ASDF2.27 was added to the *features* so you can detect the presence of this massively updated ASDF and its new features with #+asdf2.27. This is *not* ASDF3, for we painfully maintained backwards compatibility.
* Portability updates were done for ABCL, CLISP, CMUCL, ECL, LispWorks Personal Edition, MKCL, XCL.
* Documentation received some small updates.
* Tests were updated, notably including support for asdf-bundle. They were successfully run on ccl clisp sbcl ecl ecl_bytecodes cmucl abcl scl allegro lispworks allegromodern xcl. Manual tests were run on lispworks-personal-edition genera. Untested were cormancl mkcl rmcl.
* POIU was updated. This extension is distributed separately from ASDF. It is now much simplified and made more correct by construction by reusing the new TRAVERSE infrastructure of ASDF and now being able to assume the ASDF object model is a complete description of dependencies. On SBCL it will handle deferred warnings from background compilations. On platforms that cannot fork it will gracefully fall back to serial compilation.
* SLIME support for ASDF was significantly enhanced. It is distributed separately from ASDF, with SLIME. See latest checkin from 2012-12-26. We recommend that in your ~/.swank.lisp you should: (in-package :swank) (defparameter *upgrade-asdf-p* t) (pushnew 'try-compile-file-with-asdf *compile-file-for-emacs-hook*)
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org He wa'n't no common dog, he wa'n't no mongrel; he was a composite. A composite dog is a dog that is made up of all the valuable qualities that's in the dog breed — kind of a syndicate; and a mongrel is made up of all riffraff that's left over. — Mark Twain
Faré fahree@gmail.com writes:
OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
It still breaks gbbopen and fset.
It still seems to trigger the package-at-variance issue in weblocks and other projects.
If the new ASDF isn't backwards-compatible with existing projects, I'm not inclined to update the ASDF that Quicklisp fetches any time soon.
Zach
On Fri, Dec 28, 2012 at 10:55 AM, Zach Beane xach@xach.com wrote:
Faré fahree@gmail.com writes:
OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
It still breaks gbbopen and fset.
Ouch. I will work with the authors to resolve these issues. FSet clearly relies on buggy behavior for no good reason, and the author was provided a simple patch. I'll work with the authors of each failing package to get a fix in before I release.
It still seems to trigger the package-at-variance issue in weblocks and other projects.
OK, I will try to build everything with qlmapper before I issue a new release candidate.
If the new ASDF isn't backwards-compatible with existing projects, I'm not inclined to update the ASDF that Quicklisp fetches any time soon.
Indeed and for good measure. I won't consider ASDF ready for release until all maintained packages work with it.
Thanks for the testing and apologies for the brokenness.
On 12/28/12 Dec 28 -9:55 AM, Zach Beane wrote:
Faré fahree@gmail.com writes:
OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
It still breaks gbbopen and fset.
On the Mac with ACL and SBCL, the new ASDF works equally well/equally poorly (compared with the packaged ASDF) with GBBOpen from subversion: loading the system inappropriately dumps my SLIME repl into the SWANK package, and requires a call to
(cl-user::gbbopen-user)
to get into what seems like a sane state. But it seems to build fine. What's happening for you?
r
GBBopen from svn trunk r1168 works fine for me using ASDF 2.26.45, with or without the attached cleanup patch.
rlwrap sbcl --eval '(quote(#.(require "asdf")#.(in-package :asdf)#.(upgrade-asdf)#.(load "/home/tunes/quicklisp/setup.lisp")#.(progn(load-system :gbbopen))#.(cl-user::gbbopen-test)))' rlwrap sbcl --eval '(quote(#.(load "/home/tunes/cl/asdf/asdf")#.(in-package :asdf)#.(upgrade-asdf)#.(load "/home/tunes/quicklisp/setup.lisp")#.(progn(load-system :gbbopen))#.(cl-user::gbbopen-test)))'
Note: the first cleanup in the patch, with subpathname, is not really necessary and depends on ASDF 2.018 from October 2011.
What are you trying and what error are you getting?
Granted, the build infrastructure of GBBopen dates from ASDF1 days, but really, a lot of it is just reinventing things that were done I believe better and for everyone in ASDF2, and GBBopen should probably adopt and embrace it.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org A real person has two reasons for doing anything ... a good reason and the real reason.
On Fri, Dec 28, 2012 at 3:50 PM, Robert Goldman rpgoldman@sift.info wrote:
On 12/28/12 Dec 28 -9:55 AM, Zach Beane wrote:
Faré fahree@gmail.com writes:
OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
It still breaks gbbopen and fset.
On the Mac with ACL and SBCL, the new ASDF works equally well/equally poorly (compared with the packaged ASDF) with GBBOpen from subversion: loading the system inappropriately dumps my SLIME repl into the SWANK package, and requires a call to
(cl-user::gbbopen-user)
to get into what seems like a sane state. But it seems to build fine. What's happening for you?
r
Dear Zach,
: Faré OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
: Zach Beane It still breaks gbbopen and fset.
It still seems to trigger the package-at-variance issue in weblocks and other projects.
Did you actually update ASDF when you re-ran the tests? I can't reproduce the failure with either gbbopen, weblocks, or any of the previously-mentioned projects.
With weblocks, compilation does complain about a few undefined variables and functions — but not more so than ASDF 2.26 and earlier, and whatever tests there are pass for me.
I did have to fix asdf-system-connections to avoid unnecessarily recompilations (especially now that ASDF correctly propagates dependencies from system to system), but that
Do you have failure logs I can inspect?
FSet still breaks, and unhappily will until Scott applies the trivial fix I sent.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I'm only a stupid AI, but my creator is a real genius!
Faré fahree@gmail.com writes:
Dear Zach,
: Faré OK, so 2.26.45 is my new release candidate for ASDF 2.27. Please grab it and test it.
: Zach Beane It still breaks gbbopen and fset.
It still seems to trigger the package-at-variance issue in weblocks and other projects.
Did you actually update ASDF when you re-ran the tests? I can't reproduce the failure with either gbbopen, weblocks, or any of the previously-mentioned projects.
With weblocks, compilation does complain about a few undefined variables and functions — but not more so than ASDF 2.26 and earlier, and whatever tests there are pass for me.
I did have to fix asdf-system-connections to avoid unnecessarily recompilations (especially now that ASDF correctly propagates dependencies from system to system), but that
Do you have failure logs I can inspect?
I build each of the hundreds of Quicklisp systems in alphabetical order. Perhaps some state created by that mechanism is triggering the problem in a way that doesn't happen when building a system in isolation.
I'll try to create an easy-to-reproduce test case that illustrates the failures I'm seeing.
Zach
Do you have failure logs I can inspect?
I build each of the hundreds of Quicklisp systems in alphabetical order. Perhaps some state created by that mechanism is triggering the problem in a way that doesn't happen when building a system in isolation.
I'll try to create an easy-to-reproduce test case that illustrates the failures I'm seeing.
Dear Zach,
Can you teach me how to use qlmapper to run the tests myself on my home machine? This way I can try every time I do a potentially incompatible change to ASDF, and before each release.
PS: 2.26.50 fixes some issues with force and force-not, which were causing spurious attempts to rebuild sb-posix when required rather than loaded. It that was the symptom of your failures, please try again with the latest (currently 2.26.51).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org An apple every eight hours will keep three doctors away.