On 3/17/10 Mar 17 -4:57 PM, Juan Jose Garcia-Ripoll wrote:
On Wed, Mar 17, 2010 at 9:46 PM, Robert Goldman <rpgoldman@sift.info mailto:rpgoldman@sift.info> wrote:
Second, ECL's extensions are part of ASDF. It is not just a game we play to be on this list or to ship ASDF with our system. It also sucks my development time and prevents me from working on other useful things like ECL's compiler, performance, etc, but I believe on the utility of the project: being able to build and ship arbitrary ASDF systems as libraries, executables and the like, and improving integration of ASDF with systems that work better with monolithic binary files instead of thousands of small compiled files. This is a path other implementations may follow, but no one seems to care right now.
OK, we need to be much more clear about what is under discussion at any point.
If you are talking about ECL's extensions which are part of ASDF, instead of outside ASDF, then the stricture I proposed does not apply to you.
The restrictions I proposed are for the kinds of ASDF extensions that are put in individual .asd files and that are put into asdf add-on libraries. For those it is clear we cannot allow people to simply go off and rewrite core bits of ASDF. For example, if I'm writing the ASDF definition for my FLOYD-WARSHALL transitive closure library, I can't be putting into it
(defmethod PERFORM ((op load-op) (c cl-source-file)) ....)
because that means that anyone who loads my library will be loading it into a radically different environment.
The ECL extensions are obviously not this sort of thing.
It seems to me that you have two choices:
1. really fuse the ECL extensions into ASDF proper, to the extent of putting the source in there. We will still rely on some periodic testing to make sure that any incompatibilities are caught, but James Anderson's work seems like it will fix this problem.
2. keep ECL separate and provide some statement, ideally one that can be programmatically checked, about the kind of API you expect. For example, you could add a test to the test suite that would check for the absence of a method for PERFORM on the signature (T COMPONENT). That would verify that you have the method free for your own use.
I agree that a common ground should be found, but be careful with your sentences.
This is not just 'people' and this is not just 'extenders'. Either you accept this is part of ASDF and accept its growing user base, or not. If asdf-ecl.lisp is part of ASDF then any maintainer should also care for what those other components need and do, and what they are based on.
Again, if this is part of ASDF, then you are correct, you are not just "people," and these strictures wouldn't apply to you.
However, I think you have contributed to this confusion yourself, by objecting to strictures meant only for ASDF system developers, on behalf of something that you say yourself does not apply to you.
Please focus your attention on helping us base define a clear set of guidelines rather than attacking the notion of limitations on the extenders.What we do need is clear and self explained in the asdf-ecl.lisp extension and I discussed this here long long ago. If this has to be done again, then so it be.
asdf-ecl.lisp provides six new operations that have to integrate with ASDF and that can be used instead of LOAD-OP or COMPILE-OP. These operations are DLL-OP, LIB-OP, FASL-OP, monolithic versions (that is everything in one file) and a PROGRAM-OP.
These operations proceed in a simple way: they use TRAVERSE with a LOAD-OP to know what a system uses, and then they use PERFORM with a COMPILE-OP to actually compile it and from the list that TRAVERSE creates they learn the list of compiled files and build things with them (library, program, whatever)
Integration with TRAVERSE is crucial and depending on how _you_ _all_ decide that TRAVERSE should behave then we will have to adapt the code one way or another. This integration is fragile. If at some point the algorithm is randomly changed and you decide that systems should be listed before components, then we are f*ck*d.
Can you say what exactly has goofed this up for you? Note that TRAVERSE is not deterministic --- it's possible to specify multiple different orderings that satisfy the dependency specification in the ASDF file.
It's possible that what is going on is simply a bug, but I can't tell without knowing what has changed. Can you give me a concrete example? The behavior of TRAVERSE should not have changed for your application (except up to the nondeterminism to which I refer above); the changes should only appear on REcompilation.
Until now our own operations listed no dependencies other than the LOAD-OP and COMPILE-OP mentioned before. TRAVERSE was just happy with that. Right now ASDF is adding additional dependencies such as performing the same operations (DLL-OP, LIB-OP...) on the components itself. This was my source of surprise when I started this thread.
As far as I can tell, if ASDF was not doing that before, it was an unintended behavior.
If individual files are not to be subjected to these operations, would it be possible to define these operations as SYSTEM-OPERATIONS and simply define a method on PERFORM that would ensure that SYSTEM-OPERATIONS on COMPONENTS are no-ops?
asdf-ecl.lisp only defined methods on COMPONENT-DEPENDS-ON, INPUT-FILES, OUTPUT-FILES, PERFORM and OPERATION-DONE-P. All the signatures we used contain at least either OUR components our OUR operations, so in that sense they should be allowed to exist and ASDF should be careful on deciding what to do with these methods -- for instance, rewriting paths as we discussed on another thread.
I thought you were specifically referring to core methods. I see that I was not fully understanding.
Note that the protocol for writing these new operations was never clear and it has grown up with experience and by inspecting ASDF's codebase. At some point the extensions were accepted in the source tree, point at which I asked whether the integration that was done the right way or not. The lack of answer was understood as an ok.
This exhibits two problems ASDF development. The underlying problem is that people are doing this work on very thin time slices.
1. Lack of answer can never effectively be treated as "ok" --- lack of an answer is more likely to simply mean "I haven't had time to look at this." My changes to TRAVERSE went in a very long time ago, and only after a very long discussion in the mailing list. It's clear that you didn't have time to review them then, and so an issue that we thought was closed will have to be reopened.
2. The documentation is badly bit-rotted. The section on extending the ASDF object model is in particularly bad shape. It was never complete, as far as I can tell, and it has not been updated. We badly need a protocol specification stating exactly what has to be defined to make new components and operations. I can't say that I fully understand how to do this --- my experiments have often seemed to go just fine, then led to me stumbling much later on a problem that came up because some critical method was missing.
Any suggestion about how to document this protocol would be very helpful.