- asdf-devel - mailman3.common-lisp.net

Problem with adding input files...
by Robert Goldman 30 Jun '17

30 Jun '17

Recording the included files (from read-file-form and read-file-line) has so far failed to solve the problem of ASDF missing updates to the included files. I could use a little advice here. Running the test I have pushed, I tried tracing asdf/action:action-up-to-date-p. It is not called during the second invocation of (asdf:load-system "test-include") So at the moment I do not see how the flow of control works to decide whether or not it is necessary to redo the DEFINE-OP. And maybe DEFINE-OP isn't the operation that should be re-triggered by changes to the system definition file or included files? Is the right place for these new dependencies on included files DEFINE-OP or PREPARE-OP? thanks, r

2 7

Possible incompleteness in INPUT-FILES for DEFINE-OP
by Robert Goldman 30 Jun '17

30 Jun '17

I think I may have stumbled on an incompleteness in this method. Here's the test case: 1. Define a system. S1, with a version loaded from a version file; 2. Load S1 into session. 3. Hack hack hack. 4. Bump S1's version number in the version.lisp-expr file 5. Load S2 that depends on new version of S1. As far as I can tell, ASDF is simply checking the .asd file, not S1-version.lisp-expr, to see if it's stale. So my load in step 5 is failing. This is, I believe, because S1-version.lisp-expr is not in the set of INPUT-FILES for S1's DEFINE-OP. It's a little unsettling to change this on the fly, but I think the right thing is probably to push the version.lisp-expr file onto a set of ancillary dependencies when it's read. If this sounds right, I'll stick this in as a ticket, and try to find where we could inject a fix. Best, r

3 8

Random ASDF component question
by Robert Goldman 29 Jun '17

29 Jun '17

Sorry to bother you, but why is it that the BUILD-OPERATION is attached to COMPONENT instead of SYSTEM?

2 1

Re: Testing Quicklisp with ASDF
by Anton Vodonosov 26 May '17

26 May '17

1 0

Testing Quicklisp with ASDF
by Faré 24 May '17

24 May '17

Dear Anton, now that a new Quicklisp was released, can you run a set of cl-test-grid against ASDF 3.1.7, 3.2.1 and the candidate 3.3.0 (in the "plan" branch)? If you can prioritize, quick results for all these versions of ASDF on a recent SBCL are more interesting than all results on all platforms. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org An extreme optimist is a man who believes that humanity will probably survive even if it doesn't take his advice. — John McCarthy

2 1

LOGICAL-PATHNAMES output locations
by Mark Evenson 22 May '17

22 May '17

In spite of Faré's resistance to supporting LOGICAL-PATHNAMEs in ASDF, I would ask for help in using the current capabilities documented for ASDF-3.2. To wit, I would like to implement my own OUTPUT-TRANSLATIONS for LOGICAL-PATHNAMES [as is allowed (but discouraged)][1]. [1]: https://www.common-lisp.net/project/asdf/asdf.html#Using-logical-pathnames I tried to define a (massively inefficient) :BEFORE COMPILE-OP which replaces all LOGICAL-PATHNAME instances with a call to CL:TRUENAME but that doesn't seem to work: ``` (defsystem abcl/build :version "2.0.0" :description "Build ABCL from a Lisp. Downloads necessary build-time tools to local cache." :in-order-to ((test-op (test-op abcl/build/t))) :perform (compile-op :before (op c) (when (typep c 'source-file) (with-slots (absolute-pathname) c (when (typep absolute-pathname 'logical-pathname) (setf absolute-pathname (truename absolute-pathname)))))) :components ((:module package :pathname #p"SYS:build;" :components ((:file "package"))) ``` What is the right well to specify my own OUTPUT-TRANSLATIONS mechanism? The disfavor of LOGICAL-PATHANME references in DEFSYSTEM form is fully understandable from the observation that there is no conforming way to ensure that the "right" translations are in place. But, I have come up with a use case in abcl where LOGICAL-PATHNAMEs would be very useful, and since it is "my" implementation, I can guarantee that appropiate translations are in place at runtime. My use case involves locating source code referenced by ASDF in different locations depending on whether the source is being actively developed, and is on a writable filesystem, or whether the source is being loaded from a read-only JVM source, i.e. a "jar" archive. Using LOGICAL-PATHNAMEs within `abcl.asd` allows me to configure the runtime behavior quite nicely via suitable DSL parameter to the ASDF:INITIALIALIZE-SOURCE-REGISTRY. Running the [currently-in-asdf jar translation code on the SOURCE-FILE][2] is all I need to do here to get a valid output location for COMPILE-OP. [2]: https://gitlab.common-lisp.net/asdf/asdf/blob/master/output-translations.li… -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."

2 1

Plans for ASDF 3.3
by Faré 16 May '17

16 May '17

Dear Robert, dear Lispers, I'd like to know what are the release plans will be for 3.3. My branch for proper phase separation is basically ready to be merged, but I'd like to see how compatible it is with next month's Quicklisp before I can recommend releasing it upon the masses. Note that the branch is called "plan" because it started as a refactoring how asdf builds its plan, but the changes run deeper than that — 970 lines were added or modified all over, not counting those moved around — that's double the number of lines of the original ASDF. Additional questions: should the syntax-control branch be worked on for 3.3.0? 3.2.2? 3.3.1? 3.4.0? It's a low-hanging fruit to making ASDF a more robust build tool that supports working with non-trivial syntax extension. PS: Are any (preferrably younger) lispers interested in becoming proficient at ASDF maintenance? There are plenty of TODO items of all sizes and all difficulties that could use some love. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "I'm going to live forever, or die trying!" — Spider Robinson

6 22

Upgrade failures for asdf-3.1.7 and later
by James M. Lawrence 08 May '17

08 May '17

Starting with a bare lisp image (e.g --no-userinit --no-sysinit for SBCL), the following signals a "component not found" error: (load "asdf-2.26.lisp") ;;; or whatever system you want (push #p"~/quicklisp/dists/quicklisp/software/alexandria-20170227-git/" asdf:*central-registry*) (load "asdf-3.2.1.lisp") (asdf:load-system "alexandria") The first bad commit is https://github.com/fare/asdf/commit/3a9457a I hope this wasn't intentional? Ideally an old asdf version would never get loaded, but practically speaking it sometimes does (e.g. on LispWorks PE and CLISP). And if that happens, thereafter one is unable to load a system that requires asdf-3. The above case is a whittled-down version of a quickloading problem. Best, lmj

5 24

ASDF backward compatibility track record
by Faré 03 May '17

03 May '17

[Starting a new thread with different topic, as suggested by Robert.] On Wed, May 3, 2017 at 3:30 PM, Daniel Kochmański <daniel(a)turtleware.eu> wrote: > Is there any chance for specification of the "core ASDF protocol" - > subset guaranteed to not change every few years, so one may write today > systems which will be loadadable in the future? > As far as I can tell, the (API for the) core of ASDF has been remarkably stable. ASDF still supports many systems written for ASDF 1 in the early 2000s. While the manual is by no means great, especially as regards documenting the API, it does introduce the main concepts; and then every function and every class has a docstring, except for a few internals you're supposed never to use, that only have a comment. I made a pass over the entire source code last year specifically to ensure no function was left behind. (If you find a place I failed to document, please report a bug.) If there is any part of ASDF that you are struggling to understand, please discuss what it is, and I can try to document it better. As to writing systems that will be loadable in the future, I recently wrote a best practices document to help authors: https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md It can certainly be improved, but probably will be updated on a need basis, so please ask questions. That said, a lot of ugly workarounds were necessary in the bad old days of ASDF 1 (and to a lesser degree ASDF 2). Many of these workarounds have been deprecated, and many of them have ceased to work. Instead, they were replaced by simple portable alternatives that do not require poking at internals or relying on exponentially bad algorithms. A few features were removed (if-component-dep-fails, replaced by if-feature) or are still being removed (run-shell-command, being replaced by run-program). Now, there have been cases where someone did something that relied on undocumented behavior, and never contacted ASDF maintainers to either get it documented or replaced by a stable API. And then sometimes, that code breaks when I refactor and fix ASDF internals. In those cases, tough nuggies, such code *deserves* to lose. Yet, if it's in Quicklisp, I will find out the breakage, contact the authors, and submit a patch or a suggestion on what could be done instead. I've notably fixed code (such as asdf-system-connections) and .asd files for countless systems. The proximate issue that prompted this discussion is that I dropped the heroic support that ASDF had to properly upgrade from 2.x on some implementations that were less broken than others (including SBCL). That support was becoming increasingly costly to maintain and test for dwindling benefit, when all implementations have distributed ASDF 3 for years now, and ASDF 2.x has long been unsupported (except that Xach still distributes it in Quicklisp, which is a shame). These heroics included redefined gf sets, symbols to unintern, u-o-f-r-c methods, classes for which to break transient inheritance cycles during upgrade, special variables to filter, etc. (Some of it explained in Robert's and my 2010 paper.) It is hard enough to maintain that back to ASDF 3, but back to ASDF 2 after a while proved too much trouble. Most implementations already resorted to renaming away the entire ASDF package from ASDF 1 or 2, so as to let ASDF 3 create fresh packages. I recently made that "punting" the rule on all implementations, even SBCL. This was a great relief for me. If it breaks someone who somehow tries to upgrade from ASDF 2.26, I say they deserve to lose. As to what I guess may have triggered your question, one part of ASDF that has changed a lot is the bundle support (2012-2017), née asdf-ecl (2005). It was a pain to transform this cool implementation-specific hack into a portable infrastructure that truly supports all implementations, doesn't have quadratic or cubic performance degradation, and generally well with the ASDF build model rather than cheats and defeats it. Paradoxically, what started as an ECL-only extension has had plenty of issues on ECL up until the recent 3.2.1 which I hope solves them all (notably thanks to your input). The final interface while broadly similar to that of asdf-ecl, also has notable differences. Mostly, asdf-ecl did cut a lot of corners and made a few questionable choices. I believe most of these choices were probably well justified at the time of ASDF 1, when fighting to fix ASDF could have been deemed a pointless endeavor. But when making asdf/bundle out of asdf-ecl, I made some deep changes to fix the ASDF build model, at which point these asdf-ecl choices did not make sense anymore. From your point of view as ECL maintainer, you might see the result as breaking compatibility with the old asdf-ecl. From my point of view as ASDF developer, what I did was restoring compatibility with a newly fixed ASDF 3 (2013). Best regards, —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Il n'y a absolument rien à dire de ce qui échappe complètement aux mots. The utterly unspeakable is utterly irrelevant. — Faré

1 0

ASDF 3.2.1 released
by Faré 04 Apr '17

04 Apr '17

It is with great pleasure that we announce the release of ASDF 3.2.1. This new release brings increased stability to the 3.2 series with many fixes to small issues since ASDF 3.2.0 from last January. Note that we already have significant changes in the pipeline and are still intending to release a 3.3.0 "in a couple of months". We urge implementations that are currently bundling previous versions of ASDF to adopt 3.2.1 at their earliest convenience. Release 3.2.1 contains significant bug fixes on multiple platforms and implementations, and does not introduce any incompatibilities on public APIs. It builds on 3.2.0, that did contain cleanups and refactorings that broke some previously undocumented (and sometimes explicitly deprecated) internals. Those systems in Quicklisp that used or abused these internals have been fixed (notably including slime, asdf-system-connections, cffi, iolib, prove, cl-protobufs). Details on bugfixes can be found at https://launchpad.net/asdf and in the discussions in merged branches of https://gitlab.common-lisp.net/asdf/asdf Notable credits go to Robert P. Goldman for continued testing, François-René Rideau for general coding and fixing a few more bugs than he put in, to Dave Cooper for lending access to a Windows test server, and to Anton Vodonosov for repeatedly testing with cl-test-grid. Here is the changelog entry for 3.2.1, compared to 3.1.0: cl-asdf (2:3.2.1-1) unstable; urgency=low New release: * source-registry: resolve conflicts in a way compatible with Quicklisp. * Upgrade: make the upgrade logic more robust, especially on CCL. * Require-system: better normalize module vs system names on CMUCL MKCL SBCL. * Logical pathnames: fix bad-system-name warning behavior when using LPNs. * XDG: skip empty entries, for compatibility with Ubuntu * Bundles: numerous fixes for bundles especially so for ECL and MKCL. Don't try to combine .a's as it's not portable; only ever combine but .o's. Getting rid of the *load-system-operation*, now it's always load-op. * launch-program: more fixes, notably for ECL, clasp. * Deprecation: fix issues with the deprecation schedule of some functions. * Test and Release: fixes to the release process and to a few tests. * Documentation: a document describing best practices when using ASDF. -- Francois-Rene Rideau <fare(a)tunes.org> Sun, 03 Avril 2017 14:49:29 +0100

2 2