On 1 Apr 2018, at 7:57, Mark Evenson wrote:

...

On Apr 1, 2018, at 14:20, Attila Lendvai attila@lendvai.name wrote:

...

The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.

FTR, here's the history of this issue:

https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-16041982...

https://github.com/quicklisp/quicklisp-client/issues/108

Wow! Holy stale complications, batman!

Robert apparently suggested something (apparently) much simpler in

https://github.com/quicklisp/quicklisp-client/pull/128

but without any commentary from Zach on that approach.

Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the preferred way forward from ASDF’s perspective?

"From ASDF's perspective," this is all new to me, since it was filed as a bug against Quicklisp, and as far as I know, never raised as an issue for ASDF. I could use some help here:

1. What's a minimal error case using `quickload` alone? 2. What's a minimal case that arises with using ASDF as the entry point? It seemed like there was one where if Quicklisp is up and running, and you use `asdf:load-system` to load a system, this can also happen.

Something I can type into a REPL verbatim is what I would like to see.

Also, sounds like though this is an issue on all lisps, not just ABCL as the first post suggested

Communications between ASDF and QL have been difficult since Zach dropped off this list (and, to be fair, I have never joined up to read quicklisp-devel, if there is such a thing).

Best, R

OK, I think I understand now. This happens when we need quicklisp not just to load, but to install and load, defsystem depends on systems.

I just looked at Robert Dodier's bug fix, https://github.com/quicklisp/quicklisp-client/pull/128/ and I'm a little concerned that it could raise an error if `asdf::missing-requires` isn't implemented on the error condition that is signaled by ASDF (and I don't see why it would be guaranteed to be implemented there). Shouldn't this check the type of `(asdf::error-condition c)`? Or if there's something about the conditions under which this handler is invoked that guarantees that the call to `missing-requires` will not error out, I don't see it, so it's probably worthy of a comment.

My guess is that if you checked for the `error-condition` being a subtype of `missing-component` that would work in modern ASDF. I have no idea whether it would work in ASDF 2 and I'm afraid that I don't have the time for the ASDF archaeology required to figure out how to "past-proof" this code.

HtH, Best, r

On 6 Apr 2018, at 4:20, Mark Evenson wrote:

...

On Apr 2, 2018, at 18:23, Robert Goldman rpgoldman@sift.info wrote:

On 1 Apr 2018, at 7:57, Mark Evenson wrote:

On Apr 1, 2018, at 14:20, Attila Lendvai attila@lendvai.name wrote:

The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.

FTR, here's the history of this issue:

https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-16041982...

https://github.com/quicklisp/quicklisp-client/issues/108

Wow! Holy stale complications, batman!

Robert apparently suggested something (apparently) much simpler in

https://github.com/quicklisp/quicklisp-client/pull/128

but without any commentary from Zach on that approach.

Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the preferred way forward from ASDF’s perspective?

"From ASDF's perspective," this is all new to me, since it was filed as a bug against Quicklisp, and as far as I know, never raised as an issue for ASDF. I could use some help here:

• What's a minimal error case using quickload alone? • What's a minimal case that arises with using ASDF as the entry point? It seemed like there was one where if Quicklisp is up and running, and you use asdf:load-system to load a system, this can also happen. Something I can type into a REPL verbatim is what I would like to see.

Not sure how to distinguish between your two requests for quickload alone versus ASDF as an entry point

A minimal case would be the following ASDF definition

--—depends.asd---

(defsystem depends :in-order-to ((test-op (test-op "depends/t"))))

(defsystem depends/t :defsystem-depends-on (prove-asdf) :depends-on (prove) :components ((:test-file "depends-test.lisp")))

——depends-test.lisp——

(in-package :cl-user) (prove:plan 1) (prove:pass "A test that always passes") (prove:finalize)

---

(ql:quickload :depends) should pick up the depends/t secondary system to install PROVE from the network, which is needed to provide a CLOS for the TEST-FILE component.

Component "prove-asdf" not found, required by NIL 0: (CONDITIONS::CONDITIONS-ERROR :INVISIBLEP T ASDF/FIND-COMPONENT:MISSING-DEPENDENCY (:REQUIRED-BY NIL :REQUIRES "prove-asdf")) 1: (ERROR ASDF/FIND-COMPONENT:MISSING-DEPENDENCY :REQUIRED-BY NIL :REQUIRES "prove-asdf") 2: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME NIL "prove-asdf" NIL) 3: ((SUBFUNCTION 1 ASDF/PARSE-DEFSYSTEM:REGISTER-SYSTEM-DEFINITION)) …

For ASDF3 alone, as long as PROVE is installed, there is no problem.

...

Also, sounds like though this is an issue on all lisps, not just ABCL as the first post suggested

Yes, this issue effects all Common Lisp implementations. I don’t think I even mentioned ABCL in my first message, so other than being an ABCL maintainer, I don’t see how you got that impression.

...

Communications between ASDF and QL have been difficult since Zach dropped off this list (and, to be fair, I have never joined up to read quicklisp-devel, if there is such a thing).

Yes, we are certainly dealing with the resistance of Quicklisp to deprecate ASDF2 in favor of ASDF3, for which I neither really know nor want to go into the history thereof. Rather than pointing fingers, and spreading blame, I am trying to find some compromise that works for both the ASDF and Quicklisp maintainers, as without getting ql:quickload to somehow include :defsystem-depends-on declarations as recognized load dependencies in the currently stable ASDF3, it means this useful feature for ASDF extensiblity is effectively unusable for inter-system cooperation within Quicklisp.

In the January 2018 Quicklisp systems, there are 103 references to prove-asdf, so this issue effects quite a bit of the current Quicklisp distributed ecosystem for that use case alone.

As I read the Quicklisp issues and pull-requests, Quicklisp would be willing to accept a “minimally invasive” patch if it would support asdf-2.26 as well as ASDF3.

So, to put things more succintly, given the choice between Quicklisp pulls [122][] or [128][], and given that we have advanced to asdf-3.3.1 since these requests were issued, what would be the preferred manner to patch Quicklisp that would be the most forward-looking for future ASDF3 compatibility so that Quicklisp might continue to work with :DEFSYSTEM-DEPENDS-ON clauses like it used to?

On 7 Apr 2018, at 15:56, Robert Dodier wrote:

Thanks to everyone for their interest in this topic.

In reference to https://github.com/quicklisp/quicklisp-client/pull/128/, it is only a minimal stop-gap measure which I found could fix the immediate problem which I encountered. I know very little about ASDF and Quicklisp, so I certainly had no conscious intent to assert that one condition or another must prevail. Any improvements by more knowledgeable people would be welcomed by me.

All the best, Robert Dodier

A cheesy fix would simply be to wrap it in `IGNORE-ERRORS`. But it might cause errors in its present form.

This really is more a QL issue than an ASDF one (although it illustrates an issue with wrapping errors, IMO).

Best, r

On 9 Apr 2018, at 11:17, Attila Lendvai wrote:

...

A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it might cause errors in its present form.

i've learned, painfully, that indiscriminate ignore-errors will almost always bite you back (in the form of wasted debugging time), no matter how innocent they look.

...

This really is more a QL issue than an ASDF one (although it illustrates an issue with wrapping errors, IMO).

while this is true, the implementation of QL requires a reliable way to hook into the internal state of various versions of ASDF (namely, into the situation when ASDF is looking for a system, and not finding it will lead to an error without QL intervention). if i understand it correctly, this is the crux of this issue.

Sure, and I am happy to try to support this, but not to the extent of recovering a copy of ASDF 2.x and trying to run it.

The problem is that I don't know when the missing component condition was added to ASDF, and doing this right would involve checking the enclosed condition to see if it's a missing component error. I know how to do that for a modern ASDF, but I don't know how to handle ASDFs that are too old to have this condition class. And I don't fee like it's my job to think about that: I think it's perverse to continue trying to use ASDF 2.

Best, r

I think Xach and I are trying to cope with essentially the same problem: lack of resources for a community project.

On the one hand, Xach wants to stick with and old ASDF because it's a lot of trouble to maintain it, and he's afraid of breakage.

OTOH, we don't have the resources to maintain backwards-compatibility in ASDF (though we try not to break things gratuitously). Keeping ancient unmaintained software running at the cost of introducing complexity into ASDF is something I simply can't do (when I transfer maintenance to Microsoft, they'll have that covered!).

Faré has worked heroically (with admirable assistance from Anton Vodonosv) to keep from breaking libraries and offering patches, but at the end of the day, if there's no one available to merge patches into new releases of libraries, that can't be a reason to veto introduction of a fix into ASDF. If there's no one available to change `foo-test` into `foo/test` in `crusty-library.asd` that's not really our fault (and really, should people be using `crusty-library.asd` if there isn't?). Naturally, that has the potential to make Xach's life miserable. It's not my fault or his.

When there's stuff like `crusty-library` out there, there will be a certain amount of misery to be had, and there will always be a tension between QL and ASDF about who gets to enjoy it.

I wish that I could coordinate more effectively with Xach, but I really don't know what I can do to make this work better.

Best, r

On 9 Apr 2018, at 12:13, Faré wrote:

On Mon, Apr 9, 2018 at 12:22 PM, Robert Goldman rpgoldman@sift.info wrote:

...

On 9 Apr 2018, at 11:17, Attila Lendvai wrote:

...

...

A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it might cause errors in its present form.

i've learned, painfully, that indiscriminate ignore-errors will almost always bite you back (in the form of wasted debugging time), no matter how innocent they look.

...

This really is more a QL issue than an ASDF one (although it illustrates an issue with wrapping errors, IMO).

while this is true, the implementation of QL requires a reliable way to hook into the internal state of various versions of ASDF (namely, into the situation when ASDF is looking for a system, and not finding it will lead to an error without QL intervention). if i understand it correctly, this is the crux of this issue.

Sure, and I am happy to try to support this, but not to the extent of recovering a copy of ASDF 2.x and trying to run it.

The problem is that I don't know when the missing component condition was added to ASDF, and doing this right would involve checking the enclosed condition to see if it's a missing component error. I know how to do that for a modern ASDF, but I don't know how to handle ASDFs that are too old to have this condition class. And I don't fee like it's my job to think about that: I think it's perverse to continue trying to use ASDF 2.

ASDF 2.26 is totally unsupported at this point. No implementation uses anything less than 3.1.2 (the first stable release in the 3.1 series, from May 2014). Many essential packages require 3.1.x or later. It is a waste of time to try to get 2.26 running

If Quicklisp wants to be conservative, I would recommend requiring ior providing ASDF 3.1.7, which is the last in the 3.1 series and pretty stable, from March 2016, which is two years ago.

The "official" policy of ASDF was always to not support anything that had already been superseded 2 years ago or more. 3.1.7 should be the oldest supported ASDF release.

That said, what my opinions seem to be negatively taken into account by Xach, so there.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Evolution competitively selects stable cooperative patterns.