[Starting a new thread with different topic, as suggested by Robert.]

On Wed, May 3, 2017 at 3:30 PM, Daniel Kochmański daniel@turtleware.eu wrote:

Is there any chance for specification of the "core ASDF protocol" - subset guaranteed to not change every few years, so one may write today systems which will be loadadable in the future?

As far as I can tell, the (API for the) core of ASDF has been remarkably stable. ASDF still supports many systems written for ASDF 1 in the early 2000s. While the manual is by no means great, especially as regards documenting the API, it does introduce the main concepts; and then every function and every class has a docstring, except for a few internals you're supposed never to use, that only have a comment. I made a pass over the entire source code last year specifically to ensure no function was left behind. (If you find a place I failed to document, please report a bug.)

If there is any part of ASDF that you are struggling to understand, please discuss what it is, and I can try to document it better.

As to writing systems that will be loadable in the future, I recently wrote a best practices document to help authors: https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md It can certainly be improved, but probably will be updated on a need basis, so please ask questions.

That said, a lot of ugly workarounds were necessary in the bad old days of ASDF 1 (and to a lesser degree ASDF 2). Many of these workarounds have been deprecated, and many of them have ceased to work. Instead, they were replaced by simple portable alternatives that do not require poking at internals or relying on exponentially bad algorithms.

A few features were removed (if-component-dep-fails, replaced by if-feature) or are still being removed (run-shell-command, being replaced by run-program).

Now, there have been cases where someone did something that relied on undocumented behavior, and never contacted ASDF maintainers to either get it documented or replaced by a stable API. And then sometimes, that code breaks when I refactor and fix ASDF internals. In those cases, tough nuggies, such code *deserves* to lose. Yet, if it's in Quicklisp, I will find out the breakage, contact the authors, and submit a patch or a suggestion on what could be done instead. I've notably fixed code (such as asdf-system-connections) and .asd files for countless systems.

The proximate issue that prompted this discussion is that I dropped the heroic support that ASDF had to properly upgrade from 2.x on some implementations that were less broken than others (including SBCL). That support was becoming increasingly costly to maintain and test for dwindling benefit, when all implementations have distributed ASDF 3 for years now, and ASDF 2.x has long been unsupported (except that Xach still distributes it in Quicklisp, which is a shame). These heroics included redefined gf sets, symbols to unintern, u-o-f-r-c methods, classes for which to break transient inheritance cycles during upgrade, special variables to filter, etc. (Some of it explained in Robert's and my 2010 paper.) It is hard enough to maintain that back to ASDF 3, but back to ASDF 2 after a while proved too much trouble. Most implementations already resorted to renaming away the entire ASDF package from ASDF 1 or 2, so as to let ASDF 3 create fresh packages. I recently made that "punting" the rule on all implementations, even SBCL. This was a great relief for me. If it breaks someone who somehow tries to upgrade from ASDF 2.26, I say they deserve to lose.

As to what I guess may have triggered your question, one part of ASDF that has changed a lot is the bundle support (2012-2017), née asdf-ecl (2005). It was a pain to transform this cool implementation-specific hack into a portable infrastructure that truly supports all implementations, doesn't have quadratic or cubic performance degradation, and generally well with the ASDF build model rather than cheats and defeats it. Paradoxically, what started as an ECL-only extension has had plenty of issues on ECL up until the recent 3.2.1 which I hope solves them all (notably thanks to your input). The final interface while broadly similar to that of asdf-ecl, also has notable differences. Mostly, asdf-ecl did cut a lot of corners and made a few questionable choices. I believe most of these choices were probably well justified at the time of ASDF 1, when fighting to fix ASDF could have been deemed a pointless endeavor. But when making asdf/bundle out of asdf-ecl, I made some deep changes to fix the ASDF build model, at which point these asdf-ecl choices did not make sense anymore. From your point of view as ECL maintainer, you might see the result as breaking compatibility with the old asdf-ecl. From my point of view as ASDF developer, what I did was restoring compatibility with a newly fixed ASDF 3 (2013).

Best regards,

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Il n'y a absolument rien à dire de ce qui échappe complètement aux mots. The utterly unspeakable is utterly irrelevant. — Faré