OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch.
This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies.
Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented...
Additional reviews very welcome as are test results!
[Fare: is your XCVB going to handle the kinds of inter-system issues I raised in my earlier email?]
Feeling like Richard III, I can only say "A spec! A spec! My kingdom for a spec!"
Cheers, r
On 10 February 2010 18:27, Robert Goldman rpgoldman@sift.info wrote:
OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch.
A test! This requires a test case in the test suite (and not a suit in the suitcase).
This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies.
I think you do understand the right thing, and that James is wrong to want ASDF to somehow DWIM. If you want to distinguish in code into parts that are depended on and parts that are not, then split your code into two systems. System foo-interface provides the interface, with packages, macros, specials, types, classes, gfs, and anything that matters at compile-time. System foo-implementation provides the implementation, with everything else. Clients depend on foo-interface and get recompiled if the interface changes. The overall system depends on interface AND implementation.
In other words, on top of a system that does the Right Thing(tm), i.e. invalidate dependencies recursively, you can do whatever you want, though it may take some effort. On top of a broken system, you always and forever have something broken.
Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented...
Additional reviews very welcome as are test results!
No time for that right now. Hopefully tomorrow.
[Fare: is your XCVB going to handle the kinds of inter-system issues I raised in my earlier email?]
XCVB does the Right Thing of correctly invalidating any object any dependencies of which has changed. On top of that, you can do whatever you want.
Feeling like Richard III, I can only say "A spec! A spec! My kingdom for a spec!"
I think ASDF should, like XCVB, do the Right Thing, which is obvious: a dependency is a dependency. Just like any build system on earth does. Just because legacy ASDF is broken doesn't mean it should remain so. Incremental builds are broken with ASDF, and that's a shame.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] There is no excuse for a programming language without clean formal semantics. For even if your language is not designed for formal reasoning by computers, there will still be humans who'll have to reason about programs.
good morning;
what is a 'dependency'? the word makes my head spin. in a build system, three things contribute to the completion state of any given node with respect to a given operation.[1] [nb. i refrain here from using the term 'depends on'.] - any intrinsic performance. eg, for a system, the up-to-date instantiation of the system definition or for a source file, that it compiled. - the performance with respect to internal constituents. eg, for a group (that is, a system or a module), the 'components'. - the external requirements. eg, as expressed in .asd as :depends-on or :weakly-depends-on
the determination for a given node depends on the intrinsic evaluation and on evaluations delegated to constituents and requirements. the internal determination is customarily, effectively atomic. the others are unclear. in particular, whether and in which manner the completion state of external requirements is transitive is a 'completely open question'. the answer, below, reads as if the state propagates from external requirements in some selective, but unspecified, manner. otherwise changes to foo-implementation would indicate the foo-interface had changed, which would be seen by foo- user as an indication that it itself was not complete.
this needs to be made explicit. the present definitions express only the user-to-requirement propagation rules. this problem demonstrates that one must also be able to describe the inverse propagation rules. in that the description of a system must include the rules for completion state propagation for respective operations. maybe that applies to modules as well? maybe that applies even to the level of components? eg for a source file changes to function, macro, ... definitions.
--- [1]: http://github.com/lisp/de.setf.asdf.x/blob/ 285a38e4ee18e10b5ce887d8ab2dd5e2d352aa0a/asdf-x.lisp#L2152 On 2010-02-11, at 05:38 , Faré wrote:
On 10 February 2010 18:27, Robert Goldman rpgoldman@sift.info wrote:
OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch.
A test! This requires a test case in the test suite (and not a suit in the suitcase).
This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies.
I think you do understand the right thing, and that James is wrong to want ASDF to somehow DWIM. If you want to distinguish in code into parts that are depended on and parts that are not, then split your code into two systems. System foo-interface provides the interface, with packages, macros, specials, types, classes, gfs, and anything that matters at compile-time. System foo-implementation provides the implementation, with everything else. Clients depend on foo-interface and get recompiled if the interface changes. The overall system depends on interface AND implementation.
In other words, on top of a system that does the Right Thing(tm), i.e. invalidate dependencies recursively, you can do whatever you want, though it may take some effort. On top of a broken system, you always and forever have something broken.
Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented...
Additional reviews very welcome as are test results!
No time for that right now. Hopefully tomorrow.
[Fare: is your XCVB going to handle the kinds of inter-system issues I raised in my earlier email?]
XCVB does the Right Thing of correctly invalidating any object any dependencies of which has changed. On top of that, you can do whatever you want.
Feeling like Richard III, I can only say "A spec! A spec! My kingdom for a spec!"
I think ASDF should, like XCVB, do the Right Thing, which is obvious: a dependency is a dependency. Just like any build system on earth does. Just because legacy ASDF is broken doesn't mean it should remain so. Incremental builds are broken with ASDF, and that's a shame.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http:// fare.tunes.org ] There is no excuse for a programming language without clean formal semantics. For even if your language is not designed for formal reasoning by computers, there will still be humans who'll have to reason about programs.
asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
On 2/10/10 Feb 10 -10:38 PM, Faré wrote:
On 10 February 2010 18:27, Robert Goldman rpgoldman@sift.info wrote:
OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch.
A test! This requires a test case in the test suite (and not a suit in the suitcase).
Correct. We need a test where there's an inter-system dependency, we modify an upstream system and we check to see what is done.
This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies.
I think you do understand the right thing, and that James is wrong to want ASDF to somehow DWIM. If you want to distinguish in code into parts that are depended on and parts that are not, then split your code into two systems. System foo-interface provides the interface, with packages, macros, specials, types, classes, gfs, and anything that matters at compile-time. System foo-implementation provides the implementation, with everything else. Clients depend on foo-interface and get recompiled if the interface changes. The overall system depends on interface AND implementation.
In other words, on top of a system that does the Right Thing(tm), i.e. invalidate dependencies recursively, you can do whatever you want, though it may take some effort. On top of a broken system, you always and forever have something broken.
Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented...
Additional reviews very welcome as are test results!
No time for that right now. Hopefully tomorrow.
I think to be fair to James he's not saying that ASDF should DWIM, he was asking for a minimum perturbation fix.
What I took from this was that a minimum perturbation fix is to make ASDF handle the easy case (INTRA-system module dependencies), test and incorporate that fix and move on to a better fix later.
The problem with doing the INTER-system dependencies is that I don't believe that ASDF currently has the infrastructure to do The Right Thing. In order to do the right thing, each system would have to store with itself information about the state of the dependencies when it was built.
Pace Daniel H, I don't believe that timestamps on files is sufficient to do this. We have developed an API based on operation-done-p, and I don't believe that operation-done-p is sufficient for this. Timestamps on files is sufficient for make, but I'm not confident it's sufficient for ASDF. At any rate, I believe that we need to go through a process of making proposals about a protocol that will do the right thing, and then move on.
The advantage of my currently-committed patch is that, I believe, it confines itself to a single case where The Right Thing is not debatable --- I don't think *anyone* could make a case for the current ASDF behavior, nor against the behavior that I've added.
You and I are in agreement that the behavior I've added is necessary but not sufficient. I don't think the fact that the current fix fails to handle INTER-system dependencies should be allowed to keep us from fixing INTRA-system dependencies.
I don't think James's arguments mitigate against pushing my modification onto the trunk, either. Again, the argument is that the current changes are not sufficient, not that they are wrong.
Unless I hear a serious argument against doing this by the end of the day tomorrow, I will push the fix for INTRA-system module dependencies into the trunk and kill the module-depends branch.
THEN we can go ahead and figure out how to fix INTER-system dependencies.
Best, r
I think to be fair to James he's not saying that ASDF should DWIM, he was asking for a minimum perturbation fix.
No, he's rejecting an actual fix to actually broken behaviour, in favor of something that's not specified and not specifiable, whereas anything meaningful and specifiable he could want can actually be achieved, better, in a well-specified ASDF, and cannot be achieved in any way whatsoever in a broken ASDF.
The ASDF specification language can already express dependencies and non-dependencies (just you define several systems, one for interfaces and one of implementations, or whatever).
e.g. in foo.asd: (defsystem foo :depends-on (bar foo-interface foo-implementation) ...) (defsystem foo-interface :depends-on (bar-interface) ...) (defsystem foo-implementation :depends-on (foo-interface) ...)
What I took from this was that a minimum perturbation fix is to make ASDF handle the easy case (INTRA-system module dependencies), test and incorporate that fix and move on to a better fix later.
Isn't it actually simpler to fix everything once and for all?
The problem with doing the INTER-system dependencies is that I don't believe that ASDF currently has the infrastructure to do The Right Thing. In order to do the right thing, each system would have to store with itself information about the state of the dependencies when it was built.
A timestamp is all you should be needing. What else do you need, really?
Pace Daniel H, I don't believe that timestamps on files is sufficient to do this. We have developed an API based on operation-done-p, and I don't believe that operation-done-p is sufficient for this. Timestamps on files is sufficient for make, but I'm not confident it's sufficient for ASDF. At any rate, I believe that we need to go through a process of making proposals about a protocol that will do the right thing, and then move on.
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
The advantage of my currently-committed patch is that, I believe, it confines itself to a single case where The Right Thing is not debatable --- I don't think *anyone* could make a case for the current ASDF behavior, nor against the behavior that I've added.
I admit I haven't looked at it yet.
I don't think James's arguments mitigate against pushing my modification onto the trunk, either. Again, the argument is that the current changes are not sufficient, not that they are wrong.
Indeed.
Unless I hear a serious argument against doing this by the end of the day tomorrow, I will push the fix for INTRA-system module dependencies into the trunk and kill the module-depends branch.
Thanks!
THEN we can go ahead and figure out how to fix INTER-system dependencies.
I say fix it just the same.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Nostalgia isn’t what it used to be.
good morning;
this commit[1] changes the declaration for *system-definition-search- functions* in order that an upgrade modify it. would it not be better to unbind it in the "hot-upgrade" section?
--- [1]: http://common-lisp.net/gitweb?p=projects/asdf/ asdf.git;a=commitdiff;h=09fd9358e07eff9e2174a3c730b8b4994bf4e68c
On 12 February 2010 05:04, james anderson james.anderson@setf.de wrote:
good morning;
this commit[1] changes the declaration for *system-definition-search- functions* in order that an upgrade modify it. would it not be better to unbind it in the "hot-upgrade" section?
asdf.git;a=commitdiff;h=09fd9358e07eff9e2174a3c730b8b4994bf4e68c
What difference would that make?
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] A peaceful communist? Take him to a place better run than his commune. Claims he the "extra" wealth around? not peaceful. If not, no commie.
good morning;
On 2010-02-12, at 06:11 , Faré wrote:
[...]
The problem with doing the INTER-system dependencies is that I don't believe that ASDF currently has the infrastructure to do The Right Thing. In order to do the right thing, each system would have to store with itself information about the state of the dependencies when it was built.
A timestamp is all you should be needing. What else do you need, really?
Pace Daniel H, I don't believe that timestamps on files is sufficient to do this. We have developed an API based on operation-done-p, and I don't believe that operation-done-p is sufficient for this. Timestamps on files is sufficient for make, but I'm not confident it's sufficient for ASDF. At any rate, I believe that we need to go through a process of making proposals about a protocol that will do the right thing, and then move on.
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
is that not what the `operation-time` cache[1] is already there to do? (traverse x operation-done-p) reads as if module-level "done-p" is hard-coded[2] with logic, which is different than that for components in general, and which does not act on this cache. perhaps some protocol additions would work here.
anyway, wrt to the question on line 1265, the codes reads as if the answer might be that dominant operation nodes should be able to specify the particular particular sub-operations to force.
--- [1]: slot: http://common-lisp.net/gitweb?p=projects/asdf/ asdf.git;a=blob;f=asdf.lisp;h=13aa55cb10f5347b2ff5fe6476ab56d5724a54e9;h b=09fd9358e07eff9e2174a3c730b8b4994bf4e68c#l644 reference: http://common-lisp.net/gitweb?p=projects/asdf/ asdf.git;a=blob;f=asdf.lisp;h=13aa55cb10f5347b2ff5fe6476ab56d5724a54e9;h b=09fd9358e07eff9e2174a3c730b8b4994bf4e68c#l1109 assertion : http://common-lisp.net/gitweb?p=projects/asdf/ asdf.git;a=blob;f=asdf.lisp;h=13aa55cb10f5347b2ff5fe6476ab56d5724a54e9;h b=09fd9358e07eff9e2174a3c730b8b4994bf4e68c#l1324 [2] : http://common-lisp.net/gitweb?p=projects/asdf/ asdf.git;a=blob;f=asdf.lisp;h=13aa55cb10f5347b2ff5fe6476ab56d5724a54e9;h b=09fd9358e07eff9e2174a3c730b8b4994bf4e68c#l1260
: Faré
: james anderson
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
is that not what the `operation-time` cache[1] is already there to do?
Indeed.
I propose to modify operation-done-p as follows i.e. always check op-time.
(defmethod operation-done-p ((o operation) (c component)) (let ((out-files (output-files o c)) (in-files (input-files o c)) (op-time (gethash (type-of o) (component-operation-times c)))) (and op-time (cond ((and (not in-files) (not out-files))) ;; arbitrary decision: an operation that uses nothing to ;; produce nothing probably isn't doing much, t) ((not out-files) (>= op-time (apply #'max (mapcar #'safe-file-write-date in-files)))) ((not in-files) nil) (t (and (every #'probe-file in-files) (every #'probe-file out-files) (> (apply #'min (mapcar #'safe-file-write-date out-files)) (apply #'max (mapcar #'safe-file-write-date in-files))))))))
(traverse x operation-done-p) reads as if module-level "done-p" is hard-coded[2] with logic, which is different than that for components in general, and which does not act on this cache. perhaps some protocol additions would work here. anyway, wrt to the question on line 1265, the codes reads as if the answer might be that dominant operation nodes should be able to specify the particular particular sub-operations to force.
I admit traverse is a mess that I haven't gotten my head fully around yet, so I'll leave more knowledgeable people to comment.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] People are not born rich and become poor because of some dreadful phenomenon. They are born deprived of anything and become rich when they do but by the accumulated fruits of their own and their ancestors' labor: their capital.
good afternoon;
On 2010-02-12, at 15:07 , Faré wrote:
: Faré
: james anderson
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
is that not what the `operation-time` cache[1] is already there to do?
Indeed.
I propose to modify operation-done-p as follows i.e. always check op-time.
(defmethod operation-done-p ((o operation) (c component)) (let ((out-files (output-files o c)) (in-files (input-files o c)) (op-time (gethash (type-of o) (component-operation-times c)))) (and op-time (cond ((and (not in-files) (not out-files))) ;; arbitrary decision: an operation that uses nothing to ;; produce nothing probably isn't doing much, t) ((not out-files) (>= op-time (apply #'max (mapcar #'safe-file-write-date in-files)))) ((not in-files) nil) (t (and (every #'probe-file in-files) (every #'probe-file out-files) (> (apply #'min (mapcar #'safe-file-write-date out-files)) (apply #'max (mapcar #'safe-file-write-date in- files))))))))
would it not be necessary to at least compare that with some sort of time-stamp on the operation itself?
On 2/12/10 Feb 12 -8:07 AM, Faré wrote:
: Faré
: james anderson
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
is that not what the `operation-time` cache[1] is already there to do?
Indeed.
I propose to modify operation-done-p as follows i.e. always check op-time.
(defmethod operation-done-p ((o operation) (c component)) (let ((out-files (output-files o c)) (in-files (input-files o c)) (op-time (gethash (type-of o) (component-operation-times c)))) (and op-time (cond ((and (not in-files) (not out-files))) ;; arbitrary decision: an operation that uses nothing to ;; produce nothing probably isn't doing much, t) ((not out-files) (>= op-time (apply #'max (mapcar #'safe-file-write-date in-files)))) ((not in-files) nil) (t (and (every #'probe-file in-files) (every #'probe-file out-files) (> (apply #'min (mapcar #'safe-file-write-date out-files)) (apply #'max (mapcar #'safe-file-write-date in-files))))))))
(traverse x operation-done-p) reads as if module-level "done-p" is hard-coded[2] with logic, which is different than that for components in general, and which does not act on this cache. perhaps some protocol additions would work here. anyway, wrt to the question on line 1265, the codes reads as if the answer might be that dominant operation nodes should be able to specify the particular particular sub-operations to force.
I admit traverse is a mess that I haven't gotten my head fully around yet, so I'll leave more knowledgeable people to comment.
See earlier responses. The problem with this modification is the dual definition of performing an operation on a module. Operation-done-p as applied to a module really only applies to the immediate call of PERFORM on the module object --- i.e., what you do in the post-order tree traversal. It does NOT get applied when you are considering the intuitive notion of performing an operation to a module --- that is, the whole operation of performing on the module object itself /and its components/.
That whole mess needs to be teased apart before we can get a version of ASDF that will do the right thing there.
best, r
good afternoon;
That whole mess needs to be teased apart before we can get a version of ASDF that will do the right thing there.
when one has a moment, one might look at asdf.x[1]. it suggests a model for a teased-apart asdf, as well as a concrete reconstruction, which closely resembles the original. while the particular realization may not cover the eventual asdf-as-the-one-and-only functionality, an argument that it leaves out essential structural aspects might lead to useful insights.
in its terms, the present issue becomes - whether the implemented dependency-qualified method for perform does the 'right thing'? - how other dependency qualified methods would be combined? it treats them as standard around methods, but it is not clear that this is correct. - how is component-done-p to be implemented for various classes? -- abstractly for modules, v/s systems v/s source files -- concretely for systems which are interfaces v/s internal libraries v/s applications.
On 2/12/10 Feb 12 -4:48 AM, james anderson wrote:
good morning;
On 2010-02-12, at 06:11 , Faré wrote:
[...]
The problem with doing the INTER-system dependencies is that I don't believe that ASDF currently has the infrastructure to do The Right Thing. In order to do the right thing, each system would have to store with itself information about the state of the dependencies when it was built.
A timestamp is all you should be needing. What else do you need, really?
Pace Daniel H, I don't believe that timestamps on files is sufficient to do this. We have developed an API based on operation-done-p, and I don't believe that operation-done-p is sufficient for this. Timestamps on files is sufficient for make, but I'm not confident it's sufficient for ASDF. At any rate, I believe that we need to go through a process of making proposals about a protocol that will do the right thing, and then move on.
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
is that not what the `operation-time` cache[1] is already there to do? (traverse x operation-done-p) reads as if module-level "done-p" is hard-coded[2] with logic, which is different than that for components in general, and which does not act on this cache. perhaps some protocol additions would work here.
One problem is that this cache does not survive across lisp sessions, so we can't appeal to it to establish correctness when our lisp first starts up (unless we add code to populate the cache with inferred compilation times).
A second problem is that there are two meanings to performing an operation on a system, and these two meanings need to be teased apart. (I don't believe this distinction is helpful.)
One meaning is the intuitive one: to operate on a module is to operate on all of its components. The second is the less intuitive tree walking one: there is a tree node that corresponds to operating on the module proper, and we do the operation by calling PERFORM on the operation and module objects.
With respect to your point about module level done-p being hard-coded, I agree --- I discussed this in terms of the distinction between operation-done-p and traverse, in my earlier response to Faré. Note that the unhelpful dual definition of performing an operation on a module almost forces this distinction.
On 2010-02-12, at 17:17 , Robert Goldman wrote:
On 2/12/10 Feb 12 -4:48 AM, james anderson wrote:
good morning;
On 2010-02-12, at 06:11 , Faré wrote:
[...]
The problem with doing the INTER-system dependencies is that I don't believe that ASDF currently has the infrastructure to do The Right Thing. In order to do the right thing, each system would have to store with itself information about the state of the dependencies when it was built.
A timestamp is all you should be needing. What else do you need, really?
Pace Daniel H, I don't believe that timestamps on files is sufficient to do this. We have developed an API based on operation-done-p, and I don't believe that operation-done-p is sufficient for this. Timestamps on files is sufficient for make, but I'm not confident it's sufficient for ASDF. At any rate, I believe that we need to go through a process of making proposals about a protocol that will do the right thing, and then move on.
Maybe the current timestamps in ASDF are somewhat lacking, but that doesn't mean timestamps can't solve the issue. I think that one timestamp per pair (op component) should do the trick, stored as an alist (op timestamp) in a slot of the component.
is that not what the `operation-time` cache[1] is already there to do? (traverse x operation-done-p) reads as if module-level "done-p" is hard-coded[2] with logic, which is different than that for components in general, and which does not act on this cache. perhaps some protocol additions would work here.
One problem is that this cache does not survive across lisp sessions, so we can't appeal to it to establish correctness when our lisp first starts up (unless we add code to populate the cache with inferred compilation times).
one might conclude, that the (cache x done-p) semantics should be defined such that they yield the same results for respective external states across system instantiations.
A second problem is that there are two meanings to performing an operation on a system, and these two meanings need to be teased apart. (I don't believe this distinction is helpful.)
[? this parenthetical sentence and the preceding clause can be read such that they contradict each other.]
One meaning is the intuitive one: to operate on a module is to operate on all of its components. The second is the less intuitive tree walking one: there is a tree node that corresponds to operating on the module proper, and we do the operation by calling PERFORM on the operation and module objects.
once one has arrived to the point, where each "component" can concern internal, intrinsic performance, that of constituents, and that of requirements, one is able to consider for which component classes which combinations of (internal x constituent x requirement) performance have which effects on completion status. yes, that is an essential step.
With respect to your point about module level done-p being hard- coded, I agree --- I discussed this in terms of the distinction between operation-done-p and traverse, in my earlier response to Faré. Note that the unhelpful dual definition of performing an operation on a module almost forces this distinction.
yes, your #3 in that earlier message, suggests that you may also have reached the point, above.
On 2/11/10 Feb 11 -11:11 PM, Faré wrote:
I think to be fair to James he's not saying that ASDF should DWIM, he was asking for a minimum perturbation fix.
No, he's rejecting an actual fix to actually broken behaviour, in favor of something that's not specified and not specifiable, whereas anything meaningful and specifiable he could want can actually be achieved, better, in a well-specified ASDF, and cannot be achieved in any way whatsoever in a broken ASDF.
The ASDF specification language can already express dependencies and non-dependencies (just you define several systems, one for interfaces and one of implementations, or whatever).
e.g. in foo.asd: (defsystem foo :depends-on (bar foo-interface foo-implementation) ...) (defsystem foo-interface :depends-on (bar-interface) ...) (defsystem foo-implementation :depends-on (foo-interface) ...)
What I took from this was that a minimum perturbation fix is to make ASDF handle the easy case (INTRA-system module dependencies), test and incorporate that fix and move on to a better fix later.
Isn't it actually simpler to fix everything once and for all?
I don't believe so, because I think there are some subtleties, and I'd rather get this obviously busted behavior fixed right now. Also, pace the git-lovers, I don't like having a long-lived branch on the central repository, because the updates aren't clean.
The problem with doing the INTER-system dependencies is that I don't believe that ASDF currently has the infrastructure to do The Right Thing. In order to do the right thing, each system would have to store with itself information about the state of the dependencies when it was built.
A timestamp is all you should be needing. What else do you need, really?
I need to think about this a little bit more. In particular, I can think of some issues that we should push our heads through:
1. We need to understand clearly the two notions of correctness in build --- correctness within a single lisp session and correctness across lisp sessions.
2. What do we do about a general solution? I.e., how do we handle the fact that the set of operations and components is extensible? My bet is that we should systematize what it means for an operation to be done, rather than have operation-done-p just be arbitrary lisp code. I mean we should support standard definitions of operation-done-p: e.g., all input files are newer than all output files, never (as for test-op), etc. Users would be free to fall back to arbitrary
3. Currently there's a hard distinction between operation-done-p, which only works on "internal" considerations and dependency management, which is done by TRAVERSE. I don't think this distinction is particularly helpful. At any rate it is nowhere clearly specified.
I'd like to see a quasi-formal spec roughed out before we move beyond fixing the obvious bugs, and I don't have the time to do that right now (or even to review such a spec), so I'd like to get the immediate irritant --- the intra-system dependency bug --- removed now.
Cheers, r
-
Faré -
james anderson -
Robert Goldman