(Moving the discussion from the ecl list to the asdf list, originally Re: ECL support for cl-launch and xcvb)
2009/11/9 Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com:
For the record, the issue I finally found and fixed in building an image was as follows: asdf:traverse is traversing system dependencies
I have a problem with asdf:traverse. I mean, this function drives me mad. Its logic for listing what has to be done and most important _when_ is either broken or random. The last problem we had was that for certain systems traverse lists the LOAD-OP on the system far away from the LOAD-OPs on its individual components. ASDF:MAKE-BUILD was previously using the components to decide what to link, but for those systems arbitrary dependencies from other systems would be pushed in between breaking everything. Now ECL does the following: it invokes TRAVERSE with a hacked LOAD-OP method that reports NIL when queried whether that operation was than; then it takes the list and groups all subcomponents of the library that we want to turn into a big FASL, program, etc, and then it compiles and links everything. I did not change anything in TRAVERSE, but rather in how we use it.
I noticed that you're using what looks like an old version of asdf.lisp. There have been numerous improvements in asdf recently, and having been following them, I think that current revision 1.369 from git is probably a good pick for an update, except that it doesn't work in ECL as-is. It would be nice if you would merge your changes into upstream.
This is another thing I have been asking help from the ASDF maintainers. We cannot push our extensions upstream until they have been agreed upon, I mean we need a blessing for what we are doing with TRAVERSE and the like, and the assurance that the enging below is not going to arbitrary change breaking the operations we have. Again, I have written like five emails to the list without any answer. It is discouraging.
The problem is that you'll find no one to bless your changes, as ASDF does not have a clearly defined dictator, the original author having left, our main maintainer not having the resources to become one, and a vast undefined community of users not being able to provide a clear permission wrt making incompatible changes since it's not clear what in the implementation or its documentation is binding for the future and what isn't.
I propose you offer a patch, and convince people with lots of systems to try it. I'm volunteering to try your backwards-incompatible ASDF improvements on all the systems used at ITA. Possibly, the DWIM.hu guys will try it on all the systems they have, and so on. If no one complains (and why should they?) then your patch will be a strong candidate for inclusion. That automated test framework of yours is also a good thing to show that your changes are not harmful.
The other thing is that I have come to realize is that there is no systematic way of testing libraries. ASDF does not provide support for this. I have posted a call for help, or better for consensus the Common Lisp community, but so far the best I got was either being ignored or recriminated for being too emphatic about this.
Yeah, I saw your messages on the ASDF mailing-list. I think it's not going to happen there. I'd like to support something in XCVB, though, but I don't know what yet, and have been fighting more urgent lacks of feature, first.
You seem to be also confused w.r.t. what is really needed. ASDF does not have to support any single test library out there. What it needs is to ask developers to provide a valid TEST-OP, subclassing an operation it already provides. It is as simple as saying PERFORM on (YOUR-OWN-TEST-OP ANY-COMPONENT) should return two values: OK-VALUE and a SEQUENCE of failed tests inwhatever format you want.
Do you propose that people (or their libraries) should do things like that? (eval `(defmethod asdf:perform ((op asdf:test-op) (c (eql ,(find-system :foo)))) ...))
If ASDF specifies rigorously how test operations should behave then each test framework out there will be able to provide a uniform interface for testing in a couple of lines. I already showed how to do it for RT, for instance.
Sounds good.
The only complain on the ASDF side was that currently PERFORM can not return values because it is not know the order in which operations are performed and because a single operation may trigger A LOT of other operations. That is why I developed a different implementation of ASDF:OPERATE which works incrementally creating a log of operations and output values. This log is extremely valuable because 1) by filtering test operations we can know which tests fails, 2) in general it allows a completely automated analysis of what ASDF did and what operations failed, instead of the current ugly interface based on the debugger. Both features are essential for automated building and testing.
Interestingly, antifuchs' POIU also has a logging facility, to allow for replay of past builds (with non-deterministism introduced by the termination order of the parallel compilation jobs).
The patches have been submitted to the list but, as I said, no answer so far.
Yes, I saw the patch, but couldn't assume the authority to (in)validate it, and probably no one else could.
Getting a patch widely tested could reassure us that "in practice, no one relies on the behavior of the old TRAVERSE where it is incompatibly redefined by the new one".
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] One can be so anxious to put his "best foot forward" that he doesn't even notice that it isn't his own foot. -- Harry Browne (HIFFIAUW)
On Tue, Nov 10, 2009 at 3:04 AM, Faré fahree@gmail.com wrote:
The problem is that you'll find no one to bless your changes, as ASDF does not have a clearly defined dictator, the original author having left, our main maintainer not having the resources to become one, and a vast undefined community of users not being able to provide a clear permission wrt making incompatible changes since it's not clear what in the implementation or its documentation is binding for the future and what isn't.
Understood.
I propose you offer a patch, and convince people with lots of systems to try it. I'm volunteering to try your backwards-incompatible ASDF improvements on all the systems used at ITA. Possibly, the DWIM.hu guys will try it on all the systems they have, and so on. If no one complains (and why should they?) then your patch will be a strong candidate for inclusion. That automated test framework of yours is also a good thing to show that your changes are not harmful.
When I find time, in a week or so, I will try to produce a set of patches for the following things: - A new function that creates a sorted list of module dependencies wrapping around TRAVERSE This function should produce the same result irrespectively of what has been loaded or compiled. - Building on that, a new file, asdf-ecl.lisp that provides new operations for building standalone executables, fasls and other niceties with ECL. - A logging facility for ASDF, with a bit more cleaned up interface. - A test operation that builds on the logging facility and which can be completed by any test framework.
Note that my changes are *not* backwards incompatible and do *not* change TRAVERSE, but any further change that alters the TRAVERSE function and its output will break the first two add-ons.
Furthermore, in order to answer your question, what I expect of the test facility is that once every framework hooks in, one is able to write
(defsystem :my-package :components ((:file "my-package")) :in-order-to ((test-op (rt-test-op :my-package-test))))
In the mean time, one will have to resort to EQL specialized methods, though.
Juanjo
Juan Jose Garcia-Ripoll writes:
When I find time, in a week or so, I will try to produce a set of patches for the following things:
- A new function that creates a sorted list of module dependencies
wrapping around TRAVERSE This function should produce the same result irrespectively of what has been loaded or compiled.
- Building on that, a new file, asdf-ecl.lisp that provides new
operations for building standalone executables, fasls and other niceties with ECL.
- A logging facility for ASDF, with a bit more cleaned up interface.
- A test operation that builds on the logging facility and which can
be completed by any test framework.
I'll happily try anything you provide for the last two points. Last time, you said it's a proof-on-concept and didn't make it sound like you want people to test it out.
As ASDF uses git now, you can easily create your own branch, so to make it easier for people to try it.
-T.
On Tue, Nov 10, 2009 at 11:57 AM, Tobias C. Rittweiler tcr@freebits.de wrote:
Juan Jose Garcia-Ripoll writes:
When I find time, in a week or so, I will try to produce a set of patches for the following things:
- A new function that creates a sorted list of module dependencies
wrapping around TRAVERSE This function should produce the same result irrespectively of what has been loaded or compiled.
- Building on that, a new file, asdf-ecl.lisp that provides new
operations for building standalone executables, fasls and other niceties with ECL.
- A logging facility for ASDF, with a bit more cleaned up interface.
- A test operation that builds on the logging facility and which can
be completed by any test framework.
I'll happily try anything you provide for the last two points. Last time, you said it's a proof-on-concept and didn't make it sound like you want people to test it out.
I think I used the physicists view of "proof of concept" which is more like, see, it can be done, let the engineers improve it themselves :-)
As ASDF uses git now, you can easily create your own branch, so to make it easier for people to try it.
Ok, I will set up a new repo here once I have cleaned up the changes:
http://tream.dreamhosters.com/git/
Juanjo
Juan Jose Garcia-Ripoll wrote:
Furthermore, in order to answer your question, what I expect of the test facility is that once every framework hooks in, one is able to write
(defsystem :my-package :components ((:file "my-package")) :in-order-to ((test-op (rt-test-op :my-package-test))))
FWIW, I'd suggest
(defsystem my-package :class rt-testable-system :components ..... )
When you do (oos 'test-op 'my-package), the rt-testable-system (presumably a subclass of system and rt-testable-mixin) would take care of invoking RT to test the system.
[As an aside, I'm intrigued that you are using RT --- we gave up on it eventually because it's state is all global, so that we were never comfortable that stuff we set up to test in one system would not clash with tests in other systems....]
On Tue, Nov 10, 2009 at 11:34 PM, Robert Goldman rpgoldman@sift.info wrote:
FWIW, I'd suggest
(defsystem my-package :class rt-testable-system :components ..... )
When you do (oos 'test-op 'my-package), the rt-testable-system (presumably a subclass of system and rt-testable-mixin) would take care of invoking RT to test the system.
That is another approach: instead of creating new operations, creating new system classes.
[As an aside, I'm intrigued that you are using RT --- we gave up on it eventually because it's state is all global, so that we were never comfortable that stuff we set up to test in one system would not clash with tests in other systems....]
I do not have any preference for test systems, but RT is used on the only place I have to run tests: the ECL test suite, which includes the ANSI test suite, a set of regression tests and two older suites. Since each of them is ran separately, it really does not matter. However, the issue of global state can be solved with a couple lines of code -- it was done already in the patches I submitted here.
Juanjo
Robert Goldman writes:
[As an aside, I'm intrigued that you are using RT --- we gave up on it eventually because it's state is all global, so that we were never comfortable that stuff we set up to test in one system would not clash with tests in other systems....]
For my needs so far, I always grabbed rt.lisp, slurped it into my project tree, and built a domain-specific testing infrastructure on top of it. (Of course, I adapted the package of the rt code each time, so there's no problem with one shared global RT state.)
As long as you're dealing with readably-printable data structures, RT is just fine -- and it's so minimal that you can just include it and do not have to add a dependency to your project.
-T.
Faré wrote:
(Moving the discussion from the ecl list to the asdf list, originally Re: ECL support for cl-launch and xcvb)
2009/11/9 Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com:
[...snip...]
This is another thing I have been asking help from the ASDF maintainers. We cannot push our extensions upstream until they have been agreed upon, I mean we need a blessing for what we are doing with TRAVERSE and the like, and the assurance that the enging below is not going to arbitrary change breaking the operations we have. Again, I have written like five emails to the list without any answer. It is discouraging.
The problem is that you'll find no one to bless your changes, as ASDF does not have a clearly defined dictator, the original author having left, our main maintainer not having the resources to become one, and a vast undefined community of users not being able to provide a clear permission wrt making incompatible changes since it's not clear what in the implementation or its documentation is binding for the future and what isn't.
An additional problem is that there is no clear relationship between the ASDF project and the versions of ASDF shipped with implementations.
I find that this is a strong disincentive for me to work on even backwards-compatible extensions to ASDF (asdf extensions providing new functionality, but not breaking any old functionality).
The lisp code I work on is shared across a number of organizations and I don't have a good way of getting people at all these organizations to adopt a patched ASDF. That means that I have to both provide a system that employs the new functionality /and/ a version of the system (presumably using conditional reading) that will replicate the functionality that I have pushed into ASDF....
Worse than that, Classic ASDF\tm doesn't seem to have provided any versioning information for us to use in conditional compilation.
I'm inclined to think that the ASDF-using community needs to somehow evolve a committee --- ideally with representatives from the SBCL developers, CCL developers, Franz, and LispWorks --- that will help vet ASDF and deal with release processes.
This would meet the implementors need for some idea about when would be a good time to pull an ASDF revision into their CL, and would give ASDF programmers some sense that their efforts are unwasted.
r
2009/11/10 Robert Goldman rpgoldman@sift.info:
An additional problem is that there is no clear relationship between the ASDF project and the versions of ASDF shipped with implementations.
I find that this is a strong disincentive for me to work on even backwards-compatible extensions to ASDF (asdf extensions providing new functionality, but not breaking any old functionality).
The lisp code I work on is shared across a number of organizations and I don't have a good way of getting people at all these organizations to adopt a patched ASDF. That means that I have to both provide a system that employs the new functionality /and/ a version of the system (presumably using conditional reading) that will replicate the functionality that I have pushed into ASDF....
Here's the way out I use with XCVB, that depends on a quite recent copy ASDF: 1- include your own copy of ASDF in your source distribution. 2- first, obliterate the previous ASDF with /xcvb/no-asdf.
In case you can't do that (e.g. cl-launch doesn't support this overriding of ASDF), make sure your basic functionality works with a legacy ASDF and extensions are #+'ed away.
Worse than that, Classic ASDF\tm doesn't seem to have provided any versioning information for us to use in conditional compilation.
My way out is to handle that in your larger build system. ASDF needs to be setup somehow, so you need scripts that DTRT wrt loading ASDF itself and setting the central-registry. That build system can push the *features* that distinguish a good ASDF from a bad one.
Or you could start helping me develop XCVB. It is getting quite usable these days.
I'm inclined to think that the ASDF-using community needs to somehow evolve a committee --- ideally with representatives from the SBCL developers, CCL developers, Franz, and LispWorks --- that will help vet ASDF and deal with release processes.
Committee, schmommittee. If you want for a document to be born ten years from now that will standardize this ten year old technology, a committee is what you need.
If what you want is a better build system, get hacking -- and may I suggest that you hack on current technology like XCVB instead of trying to extend an evolutionary dead-end?
I think what ASDF itself needs is some minor work of 1- cleaning up, unifying and stabilizing the various versions that are available for the several implementations 2- maybe a few essential hooks for extensions (such as a flag "list as dependency of LOAD-OP'ing the parent", vs "present as dependency of other things"). 3- continuing support for legacy code, including existing ASDF plugins.
I don't think it's a good idea to try invest in making big changes or innovations on top of ths code base. I recommend you give a try to XCVB and extending it -- or not even extending it: you can already have a build /foo and another build /foo/test that depends on foo -- or better, whose individual files each depend on relevant parts of foo, which allows for incremental testing.
This would meet the implementors need for some idea about when would be a good time to pull an ASDF revision into their CL, and would give ASDF programmers some sense that their efforts are unwasted.
It's a two-way street. If we can't accommodate the legitimate needs of implementers such as those of ECL, why would they bother trying to accommodate us?
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] If money is your hope for independence you will never have it. The only real security that a man will have in this world is a reserve of knowledge, experience, and ability. -- Henry Ford
-
Faré -
Juan Jose Garcia-Ripoll -
Robert Goldman -
Tobias C. Rittweiler