Just telling that it failed isn't very useful, especially when others can't reproduce (painful with old SBCL, very expensive with ACL). Can you attach a full log of the failures? Does ACL work better with old version of the test suite? I remember that a lot of those tests were failing on clisp at least. PS: I see you were in Cambridge MA recently. Next time you are, contact me! [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Death is only a milestone - albeit one that is dropped on you from a very great height — Terry Pratchett. 2010/1/27 Robert Goldman <rpgoldman@sift.info>:

On 1/27/10 Jan 27 -12:50 AM, Faré wrote:

...

I've just released ASDF 1.501 in the official repository, now with all the source registry configuration that I previously discussed. It's currently documented in its own file README.source-registry, rather than in the general manual asdf.texinfo, as it should be. Patch welcome.

Note that I bumped the version from 1.375 to 1.500 then 1.501. This to indicate that we're not using CVS anymore, that I've reached a milestone towards my goal of an "ASDF 2" that I could push as a replacement to ASDF. It passes the tests with SBCL. But the tests could be extended to do more.

Next, comes a similar revamp of ASDF-BINARY-LOCATIONS configuration. Or maybe a wholesale replacement of ABL with something that's simpler and configured in a way similar to source-registry? What do YOU think?

I have an old copy of SBCL, 1.0.28, which I keep around (we pinned ourselves to that revision for a project I was working on), and I tried to run the test suite on this version of SBCL, 64-bit Mac.

The test suite failed, and here are the last several lines of the output:

; compilation unit finished ;   caught 2 STYLE-WARNING conditions ;   printed 1 note

; /Users/rpg/lisp/asdf/asdf.fasl written ; compilation finished in 0:00:07.450 Testuite failed: ASDF compiled with warningsbash-3.2$

I thought that this might be a spurious failure having to do with being too stringent about what constitutes an ASDF compilation failure, so I tried to run the test suite again (figuring a compiled version of asdf.lisp would now be available), but it failed identically.

Is this expected?  Should I ticket this?

I will report on ACL tests shortly.

Thanks, Robert

On 1 February 2010 10:18, Robert Goldman <rpgoldman@sift.info> wrote:

On 2/1/10 Feb 1 -12:59 AM, Tobias C. Rittweiler wrote:

...

Before bumping to 2.0 directly, I'd suggest to go through at least one phase of release candidate. The release candidate is pushed to vendors (hopefully they will adopt even though it's called release candidate), the difference is that all newly introduced APIs are marked as "Experimental" until the actual 2.0 release. So the API is time-tested for a few months. Yes. When we've got the configuration and upgrade story together, we'll do a first round of glaring bug fix and push a 1.900 to the vendors.

The thing I would most like to see before the release of ASDF 2.0 is closure on the question of how to achieve backwards compatibility. Starting with asdf 1.900 we'd push :asdf2 in the features. From then on, asdf:*asdf-version* will be guaranteed to exist.

Let's say that I want to provide a library with an ASDF system definition that will work on both "Classic" ASDF and ASDF 2.0.  How do I do this? #+asdf2

:asdf-test-op-load-op-depend No, too complex, I'd say.

Of these, I think I favor the :asdf2, but I fear it, because it means we'll end up with :asdf2.5, :asdf2.6, ... etc.  But that may be the best we can come up with. For 2.5, 2.6, etc., use a check on asdf:*asdf-version*, I'd say.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] May your desire to be correct overcome your desire to have been correct (which you were not, anyway). — Faré

On 2010-02-02, at 16:13 , Robert Goldman wrote:

[1-5] Possibly each of these should go in its own mailing list thread and/or ticket. The questions seem to be substantially policy ones, so possibly for discussion on the mailing list.

before one tickets any of these issues, it would be good if they made sense. the issue is the relation between core growth and support for configuration and upgradability.

Can we split out some issues here? I think there's a lot of good stuff in this message, but having it all grouped together may not be helpful:

1. Growth of asdf.lisp

a. desirable?

no, certainly no on this order. i discount neither the need to upgrade[1], nor the use cases for site, application, and user configuration. i am skeptical that they require a second configuration system.

On 2010-02-02, at 16:38 , Pascal J. Bourguignon wrote:

...

On 2010/02/02, at 13:46 , james anderson wrote:

...

I never had a good feeling about a configuration system which introduced itself to the world with the conclusion "Hence, all in one file", and am ever less convinced that it needs to be that way.

Are you complaining here from the developers of ASDF stand point of from the users of ASDF stand point?

from the point of view of someone who would like to be a user. of something simple.

...

My understanding is that the single-file feature is a good thing for users of ASDF: they only have to download and LOAD a single file. This is good.

i agree. my objection was to the logic rather than the utility. as it were, one criteria for my thought experiment was that the asdf.lisp should be able to stand alone. thus the probe-file contingency in the bootstrap function.[2]

b. Should we be splitting the file up more? This seems likely to be helpful to diff, but would lead to one-time merge conflicts hard to get through, so possibly should be carefully scheduled.

in the interest of simplicity, the asdf.lisp should be implement core functions only. taken to the extreme, this could be just enough to read and interpret a .asd file. which is demonstrably possible, but fails to meet the 'one-file' criteria which mr bourguignon reiterated. that is, given the pre-git version, there is little reason to take things out, but - once one has already set the goal that it bootstrap itself, little reason to put anything in.

2. :contingent-on dependency. Is this ticketed? Could we put a ticket on launchpad about this?

3. graphing utility

a. core asdf? I'm inclined to think not. b. contrib to be packaged with asdf when distributed. Seems like an appealing alternative.

4. hierarchical names: this one isn't of central interest to me, so I haven't reviewed it carefully.

the references to 2-4 (plus the patch file) were intended only to call out, that the bootstrap process was able to load a patch and integrate several extensions based on the .asd file. 2 does fill a gap in the dependency logic, but that status is not significant to this discussion.

5. asdf-output-locations

a. desirable? b. contrib versus integral c. configuration

i would very much like it to be an optional, configurable contribution rather than part of the core implementation. ------- [1] http://github.com/lisp/net.comon-lisp.asdf/blob/master/ asdf.lisp#L205 [2] http://github.com/lisp/net.comon-lisp.asdf/blob/master/ asdf.lisp#L2383

On 2010/02/02, at 13:46 , james anderson wrote:

I never had a good feeling about a configuration system which introduced itself to the world with the conclusion "Hence, all in one file", and am ever less convinced that it needs to be that way.

Are you complaining here from the developers of ASDF stand point of from the users of ASDF stand point? My understanding is that the single-file feature is a good thing for users of ASDF: they only have to download and LOAD a single file. This is good. For the developers of ASDF point of view, if it becomes inconvenient to develop in a single big source file, it shouldn't be too hard to split the sources in several files, and have an asd file to build asdf.lisp from the concatenation of the source files. -- __Pascal Bourguignon__ http://www.informatimago.com

I agree with James that a bootstrapped asdf would be elegant. Unhappily it wouldn't be practical. asdf being only one file is very useful for many reasons. It being one directory would not be too bad, but still one file is better. pjb has a nice suggestion that we could split it in many files and have some build tool concatenate those files for distribution. That would work for me, but nowhere near at the top of my todo list. As pjb says, my main concerns are about

 (a) Upgradability Making asdf self-upgradable, so we no more have the horror of having to deal with antique prepackaged asdf's.

 (b) Site and user configuration Minimizing setup complexity for non-experts.

 (c) Asdf binary locations / asdf output locations ABL was already merged into ASDF by gwking. While I think it was a generally good move, it fails (b), and I think can and should be redone better.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] What one person receives without working for, another person must work for without receiving. — Adrian Rogers

...

I agree with James that a bootstrapped asdf would be elegant. Unhappily it wouldn't be practical. asdf being only one file is very useful for many reasons.

please explain.

Distribution. Only one file to download and install anywhere, rather than having to dig through a directory, make sure things are there, etc. So far, it has been a constraint respected by ASDF. Also, while I agree that additional functionality should be moved out of ASDF and bootstrapped, I believe that the current ASDF has very little functionality that could be moved out without breaking things.

the only things with notes about upgradability were the fmakunbound and the two #+ecl adds. together about a dozen lines. Configuration is part of upgradability. If ASDF has to be told in a complex way how to upgrade, newbies won't be able to do it. Sorry, but requiring the manual configuration of *central-registry* just between two steps early in the build won't cut it.

then, once asdf is configurable, is there any reason to not take abl out of the core? YES! If ABL is out of the core, where would the FASLs for ABL itself go??? Or you would have to somehow make sure ABL is loaded and recompiled from source everytime, which also sucks. ABL is not that long, is it?

--#f "Necessity is the mother of invention" is a silly proverb. "Necessity is the mother of futile dodges" is much nearer the truth. — Alfred North Whitehead

...

Distribution. Only one file to download and install anywhere, rather than having to dig through a directory, make sure things are there, etc. So far, it has been a constraint respected by ASDF.

if the requirement is, that the single asdf.lisp load stand-alone, the experiment i posted does. if more is there, an asdf:load-op bootstraps it. if nothing else is there, nothing else loads.

aside from which, how would an uninitiate notice the difference between   curl ... and   curl ... | tar xf -

And then, this opens the way for the multiple files being out of sync, having to get md5sum of each of them when debugging an issue, etc.

what is the actual functional requirement which this constraint is intended to satisfy? Minimizing the number of moving parts to bootstrap a build system.

Two parts is more than one.

...

Also, while I agree that additional functionality should be moved out of ASDF and bootstrapped, I believe that the current ASDF has very little functionality that could be moved out without breaking things.

was the version as of 2009-06 that broken?

should one not expect even the newest of bies to understand the workings of a simple `.asd`. these are the same newbies whom one expects to configure it? Configuring it with 1.591 (yes, I already had to fix bugs in 1.590) is simple enough for a newbie to to. Configuring the 2009 version requires magic forms to be inserted in

Yes. It can't be upgraded from itself, configuring it requires too much magic. the execution stream just at the right moment. It's very hard to do. Requires magic shell scripting and/or magic startup files. Been there, done that.

how are changes to the central registry necessarily involved - unless they themselves are an aspect of the upgrade?

Where will ASDF find the upgrade? Especially considering an ASDF v1 that's been distributed with your implementation, and an ASDF v2 that you downloaded for sanity. You need to setup the registry before you have your new ASDF for sure. The horror!

...

YES! If ABL is out of the core, where would the FASLs for ABL itself go??? where did they go before? where does the asdf binary file go? I don't know about ABL before it was part of ASDF. I never used ABL. Instead I used CLC and CL-Launch, that both have to do magic setup for it all to work.

if that location were to concern me, i would ensure that a logical host is involved and map it in the same way i map all the binary- typed files. And when in the bootstrap of your system do you setup logical hosts? In between the initial (require :asdf) and the (asdf:oos 'asdf:load-op :asdf) ? That's utter craziness. You can still do it if you want. No one else but you wants to.

which, in itself, is one reason to leave abl out of the core. (you didn't ask about the world of files with names which don't conform to logical pathname syntax, but just about ABL's own files. ABL is dead. Long live AOT.

asdf.lisp is now 2589 lines long. It's twice longer than before. But it does twice as much. And that's still pretty short. I don't see why you make a big deal of it. What is it YOU are trying to optimize? [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Be careful what you set your heart on — for it will surely be yours. — James Baldwin, "Nobody Knows My Name"

On 2010-02-02, at 22:48 , Robert Goldman wrote:

On 2/2/10 Feb 2 -11:39 AM, james anderson wrote:

...

On 2010-02-02, at 18:17 , Faré wrote:

...

...

...

(c) Asdf binary locations / asdf output locations ABL was already merged into ASDF by gwking. While I think it was a generally good move, it fails (b), and I think can and should be redone better.

then, once asdf is configurable, is there any reason to not take abl out of the core?

Can you explain what is meant, exactly, by "take abl out of the core"?

none of the respective functions or variables are defined if just asdf.lisp is loaded.

For that matter, I don't know what "configurable" means in this context. "Reads an init file"? It was always configurable in the sense that I could load my asdf:*central-registry*, and I developed several utilities for configuring ASDF using only this rudimentary facility.

I'm not sure I follow this. I certainly do /not/ want to see us revert to the days when asdf-binary-locations was a separate download.

if by 'separate download' you mean a separate file, why not? if your requirement is, that it be in the same tar package, that makes sense.

I'm not trying to be a nitpicker here; I just want to avoid us talking by each other in case we don't agree about these terms.

best, r

On 2010-02-02, at 23:54 , Robert Goldman wrote:

On 2/2/10 Feb 2 -4:13 PM, james anderson wrote:

...

On 2010-02-02, at 22:48 , Robert Goldman wrote:

...

On 2/2/10 Feb 2 -11:39 AM, james anderson wrote:

...

On 2010-02-02, at 18:17 , Faré wrote:

...

...

...

(c) Asdf binary locations / asdf output locations ABL was already merged into ASDF by gwking. While I think it was a generally good move, it fails (b), and I think can and should be redone better.

then, once asdf is configurable, is there any reason to not take abl out of the core?

Can you explain what is meant, exactly, by "take abl out of the core"?

none of the respective functions or variables are defined if just asdf.lisp is loaded.

Why is this important? I can see that it /is/ important to you, but I don't follow why. [this is not intended as a snippy way of saying "this isn't important; it is a bona fide request for information.]

although i strive to be just an asdf user, there have been occasions when it didn't 'just work', whether because it was 'just plain broken' or incomplete. in which case its existence as open source made it possible for me to fix it - to wit simple patches which i submit on occasion, or to try to comprehend its workings and suggest ways to complete it - for example, the issue of dependency modes. as occasional as this has been, it causes me concern when the amount of core code doubles without sufficient cause. especially as i continue to hold the belief, that it is possible to structure the additions such that they would be loaded on demand or by configuration only and otherwise not to be perceived should one need to examine its working. in principle, it's really just a matter of general coding practice.

...

...

For that matter, I don't know what "configurable" means in this context. "Reads an init file"? It was always configurable in the sense that I could load my asdf:*central-registry*, and I developed several utilities for configuring ASDF using only this rudimentary facility.

I'm not sure I follow this. I certainly do /not/ want to see us revert to the days when asdf-binary-locations was a separate download.

if by 'separate download' you mean a separate file, why not? if your requirement is, that it be in the same tar package, that makes sense.

My requirement is that after loading asdf you should be able to do something equivalent to

(asdf:oos 'asdf:load-op :asdf-binary-locations)

or some similar thing like

(require 'asdf-binary-locations)

along the lines of my demonstration, one would reference the abl file in the asdf.asd for it to be included in the bootstrap.

and it should Just Work in the sense that A-B-L just worked --- it will tease apart binaries for different lisp implementations.

[It's more than OK with me that after that there should be some way to be more sophisticated about this, per Faré]

The reason I'm not necessarily agreeing with you is that "in the same tarball" doesn't necessarily entail "trivially loadable," although it might.

it should be possible to instruct asdf to include abl. just as to instruct asdf to leave abl out. which is different than to disable it.