- asdf-devel - mailman3.common-lisp.net

[asdf-devel] patch for component-relative-pathname
by james anderson 22 Feb '10

22 Feb '10

good morning; if a system definition root pathname contains a logical host, component-relative-pathname fails to produce pathnames which are relative in the sense that the combination with their parent's component pathname produces the intended location. this patch corrects that failing for sbcl and ccl. (defmethod component-relative-pathname ((component source-file)) (let ((*default-pathname-defaults* (component-pathname (component-parent component)))) (merge-component-relative-pathname (slot-value component 'relative-pathname) (component-name component) (source-file-type component (component-system component))))) attached below are descriptions of the respective data which illustrate the problem. in sbcl, when the result "relative" pathname is merged with that of the parent, the physical pathname host shadows the logical host. in ccl, the defaulting logic for make-pathname asserts an absolute root directory path in the "relative" pathname. sbcl: ------- 0] (rest (asdf::system-registered-p :de.setf.amqp)) #<ASDF:SYSTEM "de.setf.amqp" {11D56469}> 0] (describe *) #<ASDF:SYSTEM "de.setf.amqp" {11D56469}> [standard-object] Slots with :INSTANCE allocation: NAME = "de.setf.amqp" VERSION = "20100214-0.3" IN-ORDER-TO = ((ASDF:LOAD-OP.. DO-FIRST = ((ASDF:COMPILE-OP.. INLINE-METHODS = NIL PARENT = NIL RELATIVE-PATHNAME = #P"AMQP:" OPERATION-TIMES = #<HASH-TABLE :TEST EQL :COUNT 0 {11D9C791}> PROPERTIES = ((ASDF:SYSTEM-NICKNAMES :SETF.AMQP).. COMPONENTS = (#<ASDF:CL-SOURCE-FILE "package" {11DC6E11}>.. IF-COMPONENT-DEP-FAILS = :FAIL DEFAULT-COMPONENT-CLASS = NIL DESCRIPTION = "An AMQP client library" LONG-DESCRIPTION = "`de.setf.amqp` implements a native Common Lisp client library for the.. AUTHOR = #<unbound slot> MAINTAINER = #<unbound slot> LICENCE = #<unbound slot> SOURCE-FILE = #P"/Development/Source/dev/Library/de/ setf/amqp/amqp.asd" 0] (rest (asdf::system-registered-p :de.setf.utility)) #<ASDF:SYSTEM "de.setf.utility" {11D9C2B9}> 0] (describe *) #<ASDF:SYSTEM "de.setf.utility" {11D9C2B9}> [standard-object] Slots with :INSTANCE allocation: NAME = "de.setf.utility" VERSION = "20100214-1.0" IN-ORDER-TO = NIL DO-FIRST = ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) INLINE-METHODS = NIL PARENT = NIL RELATIVE-PATHNAME = #P"LIBRARY:DE;SETF;UTILITY;" OPERATION-TIMES = #<HASH-TABLE :TEST EQL :COUNT 0 {11DEACB9}> PROPERTIES = ((ASDF:SYSTEM-NICKNAMES "de.setf.utility")) COMPONENTS = (#<ASDF:CL-SOURCE-FILE "package" {11D9C8D9}>.. IF-COMPONENT-DEP-FAILS = :FAIL DEFAULT-COMPONENT-CLASS = NIL DESCRIPTION = #<unbound slot> LONG-DESCRIPTION = #<unbound slot> AUTHOR = #<unbound slot> MAINTAINER = #<unbound slot> LICENCE = #<unbound slot> SOURCE-FILE = #P"/Development/Source/dev/Library/de/ setf/utility/utility.asd" 0] (first (asdf::module-components (rest (asdf::system-registered- p :de.setf.utility)))) #<ASDF:CL-SOURCE-FILE "package" {11D9C8D9}> 0] (describe *) #<ASDF:CL-SOURCE-FILE "package" {11D9C8D9}> [standard-object] Slots with :INSTANCE allocation: NAME = "package" VERSION = #<unbound slot> IN-ORDER-TO = NIL DO-FIRST = ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) INLINE-METHODS = NIL PARENT = #<ASDF:SYSTEM "de.setf.utility" {11D9C2B9}> RELATIVE-PATHNAME = NIL OPERATION-TIMES = #<HASH-TABLE :TEST EQL :COUNT 0 {11DEC591}> PROPERTIES = NIL 0] (describe (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) #P"package.lisp" [structure-object] Slots with :INSTANCE allocation: HOST = #<SB-IMPL::UNIX-HOST {102CE661}> DEVICE = NIL DIRECTORY = (:RELATIVE) NAME = "package" TYPE = "lisp" VERSION = :NEWEST 0] (type-of (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) PATHNAME 0] ; ------- ccl: ------ > Error: File #P"LIBRARY:package.lisp.newest" not found > While executing: CCL::FCOMP-FIND-FILE, in process listener(1). > Type :GO to continue, :POP to abort, :R for a list of available restarts. > If continued: Skip evaluation of (asdf:operate 'asdf:load- op :de.setf.amqp.amqp-1-1-0-8-0) > Type :? for other options. 1 > (describe (rest (asdf::system-registered-p :de.setf.utility))) #<SYSTEM "de.setf.utility" #x878732E> Class: #<STANDARD-CLASS ASDF:SYSTEM> Wrapper: #<CCL::CLASS-WRAPPER ASDF:SYSTEM #x86085FE> Instance slots ASDF::NAME: "de.setf.utility" ASDF:VERSION: "20100214-1.0" ASDF::IN-ORDER-TO: NIL ASDF::DO-FIRST: ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) ASDF::INLINE-METHODS: NIL ASDF::PARENT: NIL ASDF::RELATIVE-PATHNAME: #P"LIBRARY:de;setf;utility;" ASDF::OPERATION-TIMES: #<HASH-TABLE :TEST EQL size 0/60 #x878702E> ASDF::PROPERTIES: ((ASDF:SYSTEM-NICKNAMES "de.setf.utility")) ASDF::COMPONENTS: (#<CL-SOURCE-FILE "package" #x8786E86> #<CL-SOURCE-FILE "pathnames" #x8786B1E> #<CL-SOURCE-FILE "modpackage" #x878675E> #<MODULE "documentation" #x878637E> #<CL-SOURCE-FILE "string" #x87857FE> #<CL-SOURCE-FILE "conditions" #x87853F6> #<MODULE "clos" #x8784FCE> #<MODULE "test" #x87847EE> #<CL-SOURCE-FILE "date" #x8783C56> #<CL-SOURCE-FILE "list" #x87837EE>) ASDF::IF-COMPONENT-DEP-FAILS: :FAIL ASDF::DEFAULT-COMPONENT-CLASS: NIL ASDF::DESCRIPTION: #<Unbound> ASDF::LONG-DESCRIPTION: #<Unbound> ASDF::AUTHOR: #<Unbound> ASDF::MAINTAINER: #<Unbound> ASDF::LICENCE: #<Unbound> ASDF:SOURCE-FILE: #P"/Development/Source/dev/Library/de/setf/utility/ utility.asd" 1 > (describe (first (asdf::module-components (rest (asdf::system- registered-p :de.setf.utility))))) #<CL-SOURCE-FILE "package" #x8786E86> Class: #<STANDARD-CLASS ASDF:CL-SOURCE-FILE> Wrapper: #<CCL::CLASS-WRAPPER ASDF:CL-SOURCE-FILE #x8643A8E> Instance slots ASDF::NAME: "package" ASDF:VERSION: #<Unbound> ASDF::IN-ORDER-TO: NIL ASDF::DO-FIRST: ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) ASDF::INLINE-METHODS: NIL ASDF::PARENT: #<SYSTEM "de.setf.utility" #x878732E> ASDF::RELATIVE-PATHNAME: NIL ASDF::OPERATION-TIMES: #<HASH-TABLE :TEST EQL size 0/60 #x8786B86> ASDF::PROPERTIES: NIL 1 > (describe (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) #P"LIBRARY:package.lisp.newest" Type: LOGICAL-PATHNAME Class: #<BUILT-IN-CLASS LOGICAL-PATHNAME> TYPE: (LOGICAL-PATHNAME . #<CCL::CLASS-WRAPPER LOGICAL-PATHNAME #x809E75E>) 1: (:ABSOLUTE) 2: "package" 3: "lisp" %LOGICAL-PATHNAME-HOST: "LIBRARY" %LOGICAL-PATHNAME-VERSION: :NEWEST 1 > (in-package :asdf) #<Package "ASDF"> 1 > (defmethod component-relative-pathname ((component source-file)) (let ((*default-pathname-defaults* (make-pathname :name nil :type nil :directory nil :host nil :defaults (component-pathname (component- parent component))))) (merge-component-relative-pathname (slot-value component 'relative-pathname) (component-name component) (source-file-type component (component-system component))))) #<STANDARD-METHOD COMPONENT-RELATIVE-PATHNAME (SOURCE-FILE)> 1 > (describe (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) #P"package.lisp" Type: PATHNAME Class: #<BUILT-IN-CLASS PATHNAME> TYPE: (PATHNAME . #<CCL::CLASS-WRAPPER PATHNAME #x80A0F16>) %PATHNAME-DIRECTORY: (:RELATIVE) %PATHNAME-NAME: "package" %PATHNAME-TYPE: "lisp" %PHYSICAL-PATHNAME-VERSION: :NEWEST %PHYSICAL-PATHNAME-DEVICE: NIL 1 >

3 21

[asdf-devel] Q on compiling ASDF
by Daniel Herring 22 Feb '10

22 Feb '10

Apparently the ASDF-binary-locations (now asdf-output-locations) code originated from Slime. In Slime, this code allowed a simple loader to put incompatible fasls in separate paths. Now that ABL (AOL) is a standard part of ASDF, what is the proper way to bootstrap/write a (load (compile-file "asdf.lisp")) and have the fasls appear in implementation-specific directories? I ask this in relation to Nick's problem on Lispworks. Though I suspect a few extra stack frames (close to the patch I reported for clisp) are to blame for his immediate problem, in principle it would be good to always load the compiled form. Thanks, Daniel

3 2

[asdf-devel] Module-depends patch is in place
by Robert Goldman 13 Feb '10

13 Feb '10

OK, I have merged the module-depends branch into master and pushed the result. I suppose we should probably kill the module-depends branch on the public repo now.... Have a good weekend, all! r

2 2

[asdf-devel] minor suggestions
by Tobias C. Rittweiler 13 Feb '10

13 Feb '10

* How about adding a TEST-OP to ASDF's asd? * (asdf::version-satisfies (asdf:asdf-version) (asdf:asdf-version)) may be a trivial test case for version-satisfies. It's not obvious how to add test cases. -T.

3 2

[asdf-devel] Trying again
by Robert Goldman 12 Feb '10

12 Feb '10

OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch. This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies. Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented... Additional reviews very welcome as are test results! [Fare: is your XCVB going to handle the kinds of inter-system issues I raised in my earlier email?] Feeling like Richard III, I can only say "A spec! A spec! My kingdom for a spec!" Cheers, r

3 14

[asdf-devel] Module dependencies still arguably wrong
by Robert Goldman 11 Feb '10

11 Feb '10

Earlier I claimed that the new ASDF TRAVERSE computed a conservative approximation to what needed to be recompiled. In particular, if we have (defsystem X :depends-on (Y) ....) With the new patch, if we do load-op on X, and Y has changed, X will be recompiled. In Classic ASDF, this would not happen. However, even with the new patch, this does not happen correctly. In particular, this case will not work properly (defsystem X :depends-on (Y) ...) (defsystem Z :depends-on (Y) ...) Let's imagine I load X and Z, which causes me to load Y. Now I modify some of the code of Y and reload X. ASDF will notice the change in Y and trigger a recompilation of X. BUT if I now (asdf:load-system :z), when we check the dependencies of Z and find Y, there will be nothing that needs to be done for Y, so recompilation of Z will /not/ be triggered. So the problem with trying to do a conservative estimation of what needs to be recompiled is that we don't store with a system object information about the state of things upon which it depends, so we cannot detect a change in the state of those dependencies..... Information about the last compilation of a system /is/ available in ASDF, but we don't store with X and Z information about the state of Z when they were compiled. To open YA can of worms, we also don't cache information about systems *across lisp sessions*. But I think I'll just raise that issue and then drop it like a hot potato. Cheers, r

2 2

[asdf-devel] asdf.x [was : Launchpad bug 502946
by james anderson 11 Feb '10

11 Feb '10

good morning; as is evident from the message attached below, for the past several days i have thought about things related to bug-502946, and asdf in general. my state-of-mind as of last week was, for the reasons alluded to below, that it is not productive to try to reason about system construction issues in the concrete context of asdf, -1.502, as it was at the time. i do not refer to with matters of configuration, or pathname mapping, or even 'elegant' bootstrapping, but rather to the need for a platform on which to think about and implement tools to - manage systems with articulated interfaces - manipulate system definitions programmatically - automate system definition construction and maintenance the larger concerns for integration with production, build, and deployment systems do all matter, just not right now. if one would set out the tasks in more detail, one could think about how to approach the issues, but at present, it is difficult to think even just about how asdf works. the only reasonable approach was to take asdf apart, isolate the functional components and reassemble them in a manner which allows to think about how a tool to manage program components works and how it might work. the current state is captured in de.setf.asdf.x[1]. there is a readme[2], which describes the intent, and the asdf-x.lisp[3] file, which reimplements asdf.lisp. there is also a unit test[4] which captures one interpretation of the 'strongly-dependent' requirement relation. on one hand, as a radical rational reconstruction of the original, it does not contribute to the main stream. on the other, the result provides the terms to think about what "intra-system" dependency is and a concrete context in which to implement projective proposals. cf. the commit[5] to support strong dependency by adding a state variable to the constraint machinery and adding the declaration to control it to the system definition language. most of which took the form of additional methods. --- [1]: http://github.com/lisp/de.setf.asdf.x [2]: http://github.com/lisp/de.setf.asdf.x/blob/master/README.md [3]: http://github.com/lisp/de.setf.asdf.x/blob/master/asdf-x.lisp [4]: http://github.com/lisp/de.setf.asdf.x/blob/master/test/launchpad/ bug-502946.lisp [5]: http://github.com/lisp/de.setf.asdf.x/commit/ 285a38e4ee18e10b5ce887d8ab2dd5e2d352aa0a Begin forwarded message: > From: james anderson <james.anderson(a)setf.de> > Date: 2010-02-07 23:45:56GMT+01:00 > To: rpgoldman(a)sift.info > Subject: Re: [asdf-devel] Launchpad bug 502946 > > good evening; > > you may, or may not be expecting this, but i got tired of trying to > understand traverse. > and tired of trying to understand parse-component, and defsystem, > and pathname resolution, and ... > i am now about 3/4 through rewriting asdf with clear protocols for > - definition operators (at all three levels - system, module, element) > - perform - with distinct qualifiers for requirement (dependency, > constituency) traversal, restart > - requirement and constituency specification canonicalization > - status evaluation for completion, features, other arbitrary > predicates > - settings/environment assertions per component > - pathname resolution which does not depend on the quirks of > *default-pathname-defaults* > > it's about the same amount of code - even though many of the > internal operators are decomposed to make the mechanism explicit. > > the most unsettling part is to ensure that - between the bnf in the > docs and the logic of the code, i've gotten the dependency > definitions right, but it won't be long. i've been chewing through > this since friday, so maybe by the time you're up tomorrow, there > will be some code up in github which passes the tests. the intent > is plug-compatible for the things i can understand, and i'm putting > it in a new package, so with a couple package-name swaps, one can > try things side-by-side. i will let you know. > > > On 2010-02-07, at 22:40 , Robert Goldman wrote: > >> Wrt https://bugs.launchpad.net/asdf/+bug/502946, >> module is not recompiled if _intra_-system dependency changes, I have >> spent quite some time groveling over the source code to TRAVERSE, >> and I >> believe that this is going to be quite difficult to fix. There are a >> number of assumptions baked into TRAVERSE, and some complicated >> uses of >> a variable scoped over the body of TRAVERSE, together with a >> number of >> local functions and a soupcon of unstated assumptions. >> >> I'm pretty sure that modules are substantially broken and, in the >> short >> term, I would discourage ASDF system designers from using them at >> all. >> >> I have been banging my head on this for at least a working day's >> worth >> of hours, and I need someone with whom to discuss the code, either by >> voice, IM or IRC. Email is not going to be high enough bandwidth. >> >> If anyone is interested in this enough to share the pain with me, >> please >> drop me a line and we can coordinate. >> >> Thanks, >> r >> >> _______________________________________________ >> asdf-devel mailing list >> asdf-devel(a)common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel >

1 0

[asdf-devel] last note about the module dependencies for now...
by Robert Goldman 10 Feb '10

10 Feb '10

I believe by adding a little bit more hair (a test that a component IS a module and IS NOT a system) to the TRAVERSE structure, I can salvage INTRA-system dependencies involving modules without trying to fix INTER-system dependencies, which seems like too big a bite to chew off. What would people think about that? I'm afraid, pace James A, that this won't make the code a lot simpler or more elegant, but it will make the behavior more predictable, which is a Good Thing. This exhibits a Common Lisp hassle pattern I have written myself into far too often --- you make a class (in this case system) inherit from another class (in this case module), and later on find that you should have built a triangular relationship between module, system and a common superclass. Introducing that common superclass is often quite painful.... In this case, I think backward compatibility rules out this kind of refactoring (impossible to tell whose system subclasses you might break), so the best we can do is put on some inelegant band-aids like (when (and (typep x 'module) (not (typep x 'system)) ....) cheers, r

1 0

[asdf-devel] New module-depends branch contents pushed
by Robert Goldman 10 Feb '10

10 Feb '10

Not a fast-forward push. But should be well worth it! Best, r

2 2

[asdf-devel] Refined the module dependency fix
by Robert Goldman 10 Feb '10

10 Feb '10

I have a series of patches I am about to push to the module-depends branch. Because I have had to rebase this onto master as new master changes have come in, this will not be a fast-forward update when you pull it into your local repo. So this is a head's up. I will push the update sometime after lunchtime my time (US Central time). Question: is anyone trying this out? Anyone have a good criterion for when I should close this branch out and merge it into master? Best, Robert

1 0