- asdf-devel - mailman3.common-lisp.net

[asdf-devel] Q on compiling ASDF
by Daniel Herring 22 Feb '10

22 Feb '10

Apparently the ASDF-binary-locations (now asdf-output-locations) code originated from Slime. In Slime, this code allowed a simple loader to put incompatible fasls in separate paths. Now that ABL (AOL) is a standard part of ASDF, what is the proper way to bootstrap/write a (load (compile-file "asdf.lisp")) and have the fasls appear in implementation-specific directories? I ask this in relation to Nick's problem on Lispworks. Though I suspect a few extra stack frames (close to the patch I reported for clisp) are to blame for his immediate problem, in principle it would be good to always load the compiled form. Thanks, Daniel

3 2

[asdf-devel] Module-depends patch is in place
by Robert Goldman 13 Feb '10

13 Feb '10

OK, I have merged the module-depends branch into master and pushed the result. I suppose we should probably kill the module-depends branch on the public repo now.... Have a good weekend, all! r

2 2

[asdf-devel] minor suggestions
by Tobias C. Rittweiler 13 Feb '10

13 Feb '10

* How about adding a TEST-OP to ASDF's asd? * (asdf::version-satisfies (asdf:asdf-version) (asdf:asdf-version)) may be a trivial test case for version-satisfies. It's not obvious how to add test cases. -T.

3 2

[asdf-devel] Trying again
by Robert Goldman 12 Feb '10

12 Feb '10

OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch. This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies. Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented... Additional reviews very welcome as are test results! [Fare: is your XCVB going to handle the kinds of inter-system issues I raised in my earlier email?] Feeling like Richard III, I can only say "A spec! A spec! My kingdom for a spec!" Cheers, r

3 14

[asdf-devel] Module dependencies still arguably wrong
by Robert Goldman 11 Feb '10

11 Feb '10

Earlier I claimed that the new ASDF TRAVERSE computed a conservative approximation to what needed to be recompiled. In particular, if we have (defsystem X :depends-on (Y) ....) With the new patch, if we do load-op on X, and Y has changed, X will be recompiled. In Classic ASDF, this would not happen. However, even with the new patch, this does not happen correctly. In particular, this case will not work properly (defsystem X :depends-on (Y) ...) (defsystem Z :depends-on (Y) ...) Let's imagine I load X and Z, which causes me to load Y. Now I modify some of the code of Y and reload X. ASDF will notice the change in Y and trigger a recompilation of X. BUT if I now (asdf:load-system :z), when we check the dependencies of Z and find Y, there will be nothing that needs to be done for Y, so recompilation of Z will /not/ be triggered. So the problem with trying to do a conservative estimation of what needs to be recompiled is that we don't store with a system object information about the state of things upon which it depends, so we cannot detect a change in the state of those dependencies..... Information about the last compilation of a system /is/ available in ASDF, but we don't store with X and Z information about the state of Z when they were compiled. To open YA can of worms, we also don't cache information about systems *across lisp sessions*. But I think I'll just raise that issue and then drop it like a hot potato. Cheers, r

2 2

[asdf-devel] asdf.x [was : Launchpad bug 502946
by james anderson 11 Feb '10

11 Feb '10

good morning; as is evident from the message attached below, for the past several days i have thought about things related to bug-502946, and asdf in general. my state-of-mind as of last week was, for the reasons alluded to below, that it is not productive to try to reason about system construction issues in the concrete context of asdf, -1.502, as it was at the time. i do not refer to with matters of configuration, or pathname mapping, or even 'elegant' bootstrapping, but rather to the need for a platform on which to think about and implement tools to - manage systems with articulated interfaces - manipulate system definitions programmatically - automate system definition construction and maintenance the larger concerns for integration with production, build, and deployment systems do all matter, just not right now. if one would set out the tasks in more detail, one could think about how to approach the issues, but at present, it is difficult to think even just about how asdf works. the only reasonable approach was to take asdf apart, isolate the functional components and reassemble them in a manner which allows to think about how a tool to manage program components works and how it might work. the current state is captured in de.setf.asdf.x[1]. there is a readme[2], which describes the intent, and the asdf-x.lisp[3] file, which reimplements asdf.lisp. there is also a unit test[4] which captures one interpretation of the 'strongly-dependent' requirement relation. on one hand, as a radical rational reconstruction of the original, it does not contribute to the main stream. on the other, the result provides the terms to think about what "intra-system" dependency is and a concrete context in which to implement projective proposals. cf. the commit[5] to support strong dependency by adding a state variable to the constraint machinery and adding the declaration to control it to the system definition language. most of which took the form of additional methods. --- [1]: http://github.com/lisp/de.setf.asdf.x [2]: http://github.com/lisp/de.setf.asdf.x/blob/master/README.md [3]: http://github.com/lisp/de.setf.asdf.x/blob/master/asdf-x.lisp [4]: http://github.com/lisp/de.setf.asdf.x/blob/master/test/launchpad/ bug-502946.lisp [5]: http://github.com/lisp/de.setf.asdf.x/commit/ 285a38e4ee18e10b5ce887d8ab2dd5e2d352aa0a Begin forwarded message: > From: james anderson <james.anderson(a)setf.de> > Date: 2010-02-07 23:45:56GMT+01:00 > To: rpgoldman(a)sift.info > Subject: Re: [asdf-devel] Launchpad bug 502946 > > good evening; > > you may, or may not be expecting this, but i got tired of trying to > understand traverse. > and tired of trying to understand parse-component, and defsystem, > and pathname resolution, and ... > i am now about 3/4 through rewriting asdf with clear protocols for > - definition operators (at all three levels - system, module, element) > - perform - with distinct qualifiers for requirement (dependency, > constituency) traversal, restart > - requirement and constituency specification canonicalization > - status evaluation for completion, features, other arbitrary > predicates > - settings/environment assertions per component > - pathname resolution which does not depend on the quirks of > *default-pathname-defaults* > > it's about the same amount of code - even though many of the > internal operators are decomposed to make the mechanism explicit. > > the most unsettling part is to ensure that - between the bnf in the > docs and the logic of the code, i've gotten the dependency > definitions right, but it won't be long. i've been chewing through > this since friday, so maybe by the time you're up tomorrow, there > will be some code up in github which passes the tests. the intent > is plug-compatible for the things i can understand, and i'm putting > it in a new package, so with a couple package-name swaps, one can > try things side-by-side. i will let you know. > > > On 2010-02-07, at 22:40 , Robert Goldman wrote: > >> Wrt https://bugs.launchpad.net/asdf/+bug/502946, >> module is not recompiled if _intra_-system dependency changes, I have >> spent quite some time groveling over the source code to TRAVERSE, >> and I >> believe that this is going to be quite difficult to fix. There are a >> number of assumptions baked into TRAVERSE, and some complicated >> uses of >> a variable scoped over the body of TRAVERSE, together with a >> number of >> local functions and a soupcon of unstated assumptions. >> >> I'm pretty sure that modules are substantially broken and, in the >> short >> term, I would discourage ASDF system designers from using them at >> all. >> >> I have been banging my head on this for at least a working day's >> worth >> of hours, and I need someone with whom to discuss the code, either by >> voice, IM or IRC. Email is not going to be high enough bandwidth. >> >> If anyone is interested in this enough to share the pain with me, >> please >> drop me a line and we can coordinate. >> >> Thanks, >> r >> >> _______________________________________________ >> asdf-devel mailing list >> asdf-devel(a)common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel >

1 0

[asdf-devel] last note about the module dependencies for now...
by Robert Goldman 10 Feb '10

10 Feb '10

I believe by adding a little bit more hair (a test that a component IS a module and IS NOT a system) to the TRAVERSE structure, I can salvage INTRA-system dependencies involving modules without trying to fix INTER-system dependencies, which seems like too big a bite to chew off. What would people think about that? I'm afraid, pace James A, that this won't make the code a lot simpler or more elegant, but it will make the behavior more predictable, which is a Good Thing. This exhibits a Common Lisp hassle pattern I have written myself into far too often --- you make a class (in this case system) inherit from another class (in this case module), and later on find that you should have built a triangular relationship between module, system and a common superclass. Introducing that common superclass is often quite painful.... In this case, I think backward compatibility rules out this kind of refactoring (impossible to tell whose system subclasses you might break), so the best we can do is put on some inelegant band-aids like (when (and (typep x 'module) (not (typep x 'system)) ....) cheers, r

1 0

[asdf-devel] New module-depends branch contents pushed
by Robert Goldman 10 Feb '10

10 Feb '10

Not a fast-forward push. But should be well worth it! Best, r

2 2

[asdf-devel] Refined the module dependency fix
by Robert Goldman 10 Feb '10

10 Feb '10

I have a series of patches I am about to push to the module-depends branch. Because I have had to rebase this onto master as new master changes have come in, this will not be a fast-forward update when you pull it into your local repo. So this is a head's up. I will push the update sometime after lunchtime my time (US Central time). Question: is anyone trying this out? Anyone have a good criterion for when I should close this branch out and merge it into master? Best, Robert

1 0

[asdf-devel] Looking for some help crafting a test case
by Robert Goldman 10 Feb '10

10 Feb '10

I believe my first ASDF patch had a minor buglet that could cause over-compilation. Here's the deal: when new TRAVERSE detects that the components of a module need to have an operation performed on them, it binds a dynamic variable and calls itself on the components of the module. The dynamic variable forces the components to be done even when they satisfy operation-done-p. This has an effect equivalent to propagating the module's dependencies onto the children. However, this also means that the dynamic variable is in place when we call TRAVERSE recursively on the dependencies recorded on the children, then those dependencies are forced to happen as well. In /theory/ this could cause over-compilation, over-loading, etc. However, I cannot come up with a test case where this will /actually/ occur. Why is that? It's because components of modules cannot "reach out" of the containing module to depend on anything in their surroundings. E.g. if I have a module FOO, then components of FOO can record dependencies only onto other components of FOO. This is a consequence of how name resolution works in ASDF. So I have what I think is a fix for performing unnecessary operations in ASDF, but I cannot figure out a way to craft a test case where unnecessary operations could actually occur. I feel like I should put the patch in anyway, in case we change the rules on name-lookup in ASDF, but this feels wrong. Any suggestions? Have I missed something? Is there any way, for example, a component of FOO could depend on an outside system? I don't believe that this is possible, but I could be wrong. thanks, r

2 1