Copying this conversation to asdf-devel, since these are topics of interest to any hacker who would like to take over development of ASDF.
On Mon, Jan 6, 2014 at 11:07 AM, Robert Goldman rpgoldman@sift.net wrote:
I'm dribbing and drabbing these out, I'm afraid....
FIND-OPERATION vs. MAKE-OPERATION: I'm a bit confused by this. I see that operations are all indexed in the *OPERATIONS* hash table, but although the names suggest that MAKE-OPERATION would make a fresh OPERATION object, and FIND-OPERATION would require that there be a previously existing one, it seems like they *both* address the hash table. Nevertheless, a careful distinction is made between calls, suggesting that I am missing an important distinction. It looks like there's a future interest in recovering initargs by using the CONTEXT argument, but for now that seems to be ignored (i.e., the CONTEXT argument never provides anything).
PERFORM: I believe that the ASDF protocol expects that PERFORM will not return anything of interest, correct? I don't see anyone catching return values. I figure I should probably document this as part of any documentation overhaul.
I'm glad you ask.
This is another unresolved design issues tracing back to ASDF 1, and one that I wasn't fully aware of until I wrote ASDF 3, at which point I fumbled around and ended up settling for a compromise.
The problem at heart is that * on the one hand, ASDF seems to allow operations to be parametrized by user-defined slots * on the other hand, when computing the plan, ASDF drops any such parameters as it walks the graph and discards any action such that an action of same type on same component has already been visited.
When rewriting ASDF, I tried to reconcile the two, by memoizing operations created with the same initargs, but eventually failed by fear of breaking backward compatibility: * users can and often do call make-instance 'some-operation, at which point you don't have the unique operation object. * I tried to have a per-traversal cache, by making the initial operation given to traverse be the "parent" of all other operations in a traversal; but this required incompatible changes in the component-operation-time / mark-operation-done protocol (not a big deal, since no one should be looking inside), this most importantly fails to work when one tries to load-system the second time around, and since no operation object is the same that second time, every action looks like it hasn't been done yet, and asdf loads everything the second time around. * I eventually settled for a global cache of *operations*, but got discouraged before to check with every user inside (and outside?!) quicklisp that I could indeed make the incompatible change in component-operation-time. Also, every single function or method that can be called with a user-provided operation object must now "canonicalize" the object before to use it in any of these comparisons (or more likely, the comparison function should be abstracted, and do the canonicalization, and for decent speed, cache the canonicalized operation in a slot of each compared operation object). * Finally, there was the issue of what it meant for two initargs to be equivalent — that totally depends on the operation class, and there's no protocol to help determine when that is or isn't the case. And this is further complicated by the fact that operate historically passes its keyword arguments to the operation, which includes things such as force, which are actually arguments to the plan, and shouldn't affect the operation — except that swank and gbbopen somehow used that slot (though they have both been weened off it recently, so we could leave a compatibility nil method in backward-interface and be done with it).
So there, I did an incomplete job here, and I apologize for it. It will be your call to either fully allow operation initargs, or definitely deprecate them. Obviously, no one is relying on the feature right now, since it has never worked (not uncommon a thing for ASDF — :depends-on (:feature featr component), :force (sys1 sys2), and probably more are ASDF 1 features that never worked until I fixed them, that plus the module dependency that you fixed).
PERFORM returns nothing of interest indeed — it either succeeds or signals an error.
Now, my plans for an ASDF 4 (or at least 3.2) were to move ASDF towards a cross-compilation model, which would make life easier for MOCL, but would also bring to the ASDF world most of the goodies of XCVB: off-image compilation, deterministic build, parallel build, etc. This would mean deprecating PERFORM, instead having a method PERFORM-FORM or such that returns some source code to be run on the target slave (as opposed to the planning master — the slave doesn't run ASDF, only UIOP, and doesn't compute the entire plan, so can be started quickly on N machines or cpus) — and maybe a list of dependencies to be loaded on the target. When no method is defined for PERFORM-FORM, ASDF4 would check for a PERFORM method, in which case it would load ASDF on the target as well as some set of dependencies to be determined (e.g. the defsystem-depends-on of the current system), and then would either import the entire set of system definitions being used or create a fake trivial system with one entry, and then call perform on the chosen operation and component.
Depending on how serious you are about it, you may or may not also adopt XCVB's model of :compile-depends-on, :load-depends-on :run-depends-on, with :depends-on foo implying a :compile-depends-on foo.cfasl (assuming you also support cfasls), and :load-depends-on foo.fasl (must be loaded before) and :run-depends-on foo.fasl (can be loaded after with circular dependencies), whereas :defsystem-depends-on foo implies :compile-depends-on foo.fasl and :load-depends-on foo.fasl. Or something more complex; cross-compilation is hard.
That's also advanced hacking, and a somewhat incompatible change (modulo that fallback strategy) that requires a lot of social work for a smooth transition. But hopefully, you can convince the MOCL guy to do the social work, since it's in his interest to promote a standardized cross-compilation solution, and good marketing for him (Cc'ed).
Regards,
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Live as if you were living already for the second time and as if you had acted the first time as wrongly as you are about to act now! — Viktor Frankl, "Man's Search for Meaning"