On Sun, Jan 6, 2013 at 3:26 PM, Robert Goldman rpgoldman@sift.info wrote:
This one I think is relevant to the proposed reorganization of ASDF to have changes in upstream systems trigger recompilation in systems that depend on them.
I have made a crude component which is a MADE-FILE. The made-file is built by invoking make to produce some output file.
MADE-FILE's compile-op OPERATION-DONE-P method always returns NIL, because ASDF does not understand what files the made file depends on -- that information is buried in the makefile. We don't want to duplicate that logic in ASDF. If we always invoke make, it will do the Right Thing and only rebuild the file when necessary.
*Unfortunately*, any components in a system that have a :depends-on relationship to a MADE-FILE, will always be recompiled, whether necessary or not. If I remember correctly, the logic in traversal is that you do the child if its input files are fresher than the output file *or* if one of its dependencies was run.
Now, with current ASDF, I can work around this for MADE-FILEs with stable APIs: I just shove them out into systems of their own, so they don't trigger constant recompilations of systems that depend on them.
But this behavior will be changing with the new ASDF.
I don't *believe* the existing controls on :force will solve this problem, either. We can't control this by simply saying "recompile everything if any upstream system changes" or "don't recompile anything based on upstream system changes" (although the latter will restore the current buggy behavior).
I can think of two solutions:
- KLUDGY -- allow individual system definitions (like my MADE-FILE
wrapper systems) to specify that they do not propagate recompilation forcing onto their dependents.
- BETTER, but bigger effect on the protocol: Separate the question of
whether an operation is performed on a dependency from whether that dependency propagates change downstream. We could do this by adding a function that is run to test whether an <operation,component> pair propagates forcing onto its dependents. By default it would be the same as "did the operation get performed?" as now, but it could be overridden by component classes like MADE-FILE where, e.g., I could say "did the file-write-date of the OUTPUT-FILES change?"
Does this make sense?
cheers, r
OK, so since it's already working, I assume it's respecting the constraints of ASDF, which is that planning happens before any Lisp computation.
I see two cases: 1- The computation that creates your MADE-FILE doesn't depend on any Lisp computation. Then you could either 1a- systematically perform this Make computation BEFORE the Lisp build, and have OPERATION-DONE-P return T, or 1b- (that's ugly) have this computation does during OPERATION-DONE-P itself, and return T or NIL depending on whether things have changed. Actually, I don't recommend that at all: since OPERATION-DONE-P is called *after* OUTPUT-FILES timestamps have been checked, this would be bad. But maybe that's a bug in ASDF, and we should run OPERATION-DONE-P first (unless we are JUST-DONE). 2- The computation that creates your MADE-FILE *does* depend on some previous Lisp computation. Then the only solution is to have this computation be a deterministic product of files that you can all declare as COMPONENTs of your system as STATIC-FILEs if need be, and/or have its INPUT-FILES return (a superset of) the set files from which it will be computed. Then OPERATION-DONE-P can just return T.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Any sufficiently advanced misquotation is indistinguishable from an original statement. — John McCarthy, misquoted