February 2010 - asdf-devel - mailman3.common-lisp.net

[asdf-devel] [PATCH] add &allow-other-keys to LOAD-SYSTEM, COMPILE-SYSTEM, TEST-SYSTEM
by Tobias C. Rittweiler 24 Feb '10

24 Feb '10

Although the way to extend operations by additional initargs is somewhat cumbersome, it's possible. Unfortunately, the sugar forms LOAD-SYSTEM, COMPILE-SYSTEM, and TEST-SYSTEM do not take additional initargs. -T. diff --git a/asdf.lisp b/asdf.lisp index 92e7f6e..3f230d5 100644 --- a/asdf.lisp +++ b/asdf.lisp @@ -1670,18 +1670,24 @@ created with the same initargs as the original one. (setf (documentation 'operate 'function) operate-docstring)) -(defun load-system (system &rest args &key force (verbose t) version) - "Shorthand for `(operate 'asdf:load-op system)`. See [operate][] for details." +(defun load-system (system &rest args &key force (verbose t) version + &allow-other-keys) + "Shorthand for `(operate 'asdf:load-op system)`. See [operate][] for +details." (declare (ignore force verbose version)) (apply #'operate 'load-op system args)) -(defun compile-system (system &rest args &key force (verbose t) version) - "Shorthand for `(operate 'asdf:compile-op system)`. See [operate][] for details." +(defun compile-system (system &rest args &key force (verbose t) version + &allow-other-keys) + "Shorthand for `(operate 'asdf:compile-op system)`. See [operate][] +for details." (declare (ignore force verbose version)) (apply #'operate 'compile-op system args)) -(defun test-system (system &rest args &key force (verbose t) version) - "Shorthand for `(operate 'asdf:test-op system)`. See [operate][] for details." +(defun test-system (system &rest args &key force (verbose t) version + &allow-other-keys) + "Shorthand for `(operate 'asdf:test-op system)`. See [operate][] for +details." (declare (ignore force verbose version)) (apply #'operate 'test-op system args))

3 10

[asdf-devel] upgradability with ECL
by Tobias C. Rittweiler 24 Feb '10

24 Feb '10

Current ECL HEAD comes with asdf version "1.604", but trying to load upstreams asdf.lisp won't work because of "A package with the name ASDF-EXTENSIONS already exists." Indeed, CLHS DEFPACKAGE says If one of the supplied :nicknames already refers to an existing package, an error of type package-error is signaled. In fact, using DEFPACKAGE for upgradability purposes is ill-conceived from the beginning because of: If defined-package-name already refers to an existing package, the name-to-package mapping for that name is not changed. If the new definition is at variance with the current state of that package, the consequences are undefined; ... And, e.g. the :EXPORT list changed in numerous points in past, and is not excluded from changing in the future. The portable venue would be not to use DEFPACKAGE but either to define a DEFINE-PACKAGE macro on top of the functional layer yourself, or using the functional layer directly. Mwahaha. Suckage. :-) -T.

4 4

[asdf-devel] default source registry...
by Faré 22 Feb '10

22 Feb '10

Should we base our search path on the XDG Base Directory Specification? http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html To the point of actually using getenv to get these search paths? I say we should. The default configuration files should be $XDG_CONFIG_DIRS/common-lisp/source-registry.conf $XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d/ And the default search path should include $XDG_DATA_DIRS/common-lisp/systems/ $XDG_DATA_DIRS/common-lisp/source// The default cache path should be under $XDG_CACHE_HOME/common-lisp/ Unless anyone objects, I'll implement that behavior and commit something like ASDF 1.695. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Vision is the art of seeing what is invisible to others. — Jonathan Swift

4 4

[asdf-devel] patch for component-relative-pathname
by james anderson 22 Feb '10

22 Feb '10

good morning; if a system definition root pathname contains a logical host, component-relative-pathname fails to produce pathnames which are relative in the sense that the combination with their parent's component pathname produces the intended location. this patch corrects that failing for sbcl and ccl. (defmethod component-relative-pathname ((component source-file)) (let ((*default-pathname-defaults* (component-pathname (component-parent component)))) (merge-component-relative-pathname (slot-value component 'relative-pathname) (component-name component) (source-file-type component (component-system component))))) attached below are descriptions of the respective data which illustrate the problem. in sbcl, when the result "relative" pathname is merged with that of the parent, the physical pathname host shadows the logical host. in ccl, the defaulting logic for make-pathname asserts an absolute root directory path in the "relative" pathname. sbcl: ------- 0] (rest (asdf::system-registered-p :de.setf.amqp)) #<ASDF:SYSTEM "de.setf.amqp" {11D56469}> 0] (describe *) #<ASDF:SYSTEM "de.setf.amqp" {11D56469}> [standard-object] Slots with :INSTANCE allocation: NAME = "de.setf.amqp" VERSION = "20100214-0.3" IN-ORDER-TO = ((ASDF:LOAD-OP.. DO-FIRST = ((ASDF:COMPILE-OP.. INLINE-METHODS = NIL PARENT = NIL RELATIVE-PATHNAME = #P"AMQP:" OPERATION-TIMES = #<HASH-TABLE :TEST EQL :COUNT 0 {11D9C791}> PROPERTIES = ((ASDF:SYSTEM-NICKNAMES :SETF.AMQP).. COMPONENTS = (#<ASDF:CL-SOURCE-FILE "package" {11DC6E11}>.. IF-COMPONENT-DEP-FAILS = :FAIL DEFAULT-COMPONENT-CLASS = NIL DESCRIPTION = "An AMQP client library" LONG-DESCRIPTION = "`de.setf.amqp` implements a native Common Lisp client library for the.. AUTHOR = #<unbound slot> MAINTAINER = #<unbound slot> LICENCE = #<unbound slot> SOURCE-FILE = #P"/Development/Source/dev/Library/de/ setf/amqp/amqp.asd" 0] (rest (asdf::system-registered-p :de.setf.utility)) #<ASDF:SYSTEM "de.setf.utility" {11D9C2B9}> 0] (describe *) #<ASDF:SYSTEM "de.setf.utility" {11D9C2B9}> [standard-object] Slots with :INSTANCE allocation: NAME = "de.setf.utility" VERSION = "20100214-1.0" IN-ORDER-TO = NIL DO-FIRST = ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) INLINE-METHODS = NIL PARENT = NIL RELATIVE-PATHNAME = #P"LIBRARY:DE;SETF;UTILITY;" OPERATION-TIMES = #<HASH-TABLE :TEST EQL :COUNT 0 {11DEACB9}> PROPERTIES = ((ASDF:SYSTEM-NICKNAMES "de.setf.utility")) COMPONENTS = (#<ASDF:CL-SOURCE-FILE "package" {11D9C8D9}>.. IF-COMPONENT-DEP-FAILS = :FAIL DEFAULT-COMPONENT-CLASS = NIL DESCRIPTION = #<unbound slot> LONG-DESCRIPTION = #<unbound slot> AUTHOR = #<unbound slot> MAINTAINER = #<unbound slot> LICENCE = #<unbound slot> SOURCE-FILE = #P"/Development/Source/dev/Library/de/ setf/utility/utility.asd" 0] (first (asdf::module-components (rest (asdf::system-registered- p :de.setf.utility)))) #<ASDF:CL-SOURCE-FILE "package" {11D9C8D9}> 0] (describe *) #<ASDF:CL-SOURCE-FILE "package" {11D9C8D9}> [standard-object] Slots with :INSTANCE allocation: NAME = "package" VERSION = #<unbound slot> IN-ORDER-TO = NIL DO-FIRST = ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) INLINE-METHODS = NIL PARENT = #<ASDF:SYSTEM "de.setf.utility" {11D9C2B9}> RELATIVE-PATHNAME = NIL OPERATION-TIMES = #<HASH-TABLE :TEST EQL :COUNT 0 {11DEC591}> PROPERTIES = NIL 0] (describe (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) #P"package.lisp" [structure-object] Slots with :INSTANCE allocation: HOST = #<SB-IMPL::UNIX-HOST {102CE661}> DEVICE = NIL DIRECTORY = (:RELATIVE) NAME = "package" TYPE = "lisp" VERSION = :NEWEST 0] (type-of (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) PATHNAME 0] ; ------- ccl: ------ > Error: File #P"LIBRARY:package.lisp.newest" not found > While executing: CCL::FCOMP-FIND-FILE, in process listener(1). > Type :GO to continue, :POP to abort, :R for a list of available restarts. > If continued: Skip evaluation of (asdf:operate 'asdf:load- op :de.setf.amqp.amqp-1-1-0-8-0) > Type :? for other options. 1 > (describe (rest (asdf::system-registered-p :de.setf.utility))) #<SYSTEM "de.setf.utility" #x878732E> Class: #<STANDARD-CLASS ASDF:SYSTEM> Wrapper: #<CCL::CLASS-WRAPPER ASDF:SYSTEM #x86085FE> Instance slots ASDF::NAME: "de.setf.utility" ASDF:VERSION: "20100214-1.0" ASDF::IN-ORDER-TO: NIL ASDF::DO-FIRST: ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) ASDF::INLINE-METHODS: NIL ASDF::PARENT: NIL ASDF::RELATIVE-PATHNAME: #P"LIBRARY:de;setf;utility;" ASDF::OPERATION-TIMES: #<HASH-TABLE :TEST EQL size 0/60 #x878702E> ASDF::PROPERTIES: ((ASDF:SYSTEM-NICKNAMES "de.setf.utility")) ASDF::COMPONENTS: (#<CL-SOURCE-FILE "package" #x8786E86> #<CL-SOURCE-FILE "pathnames" #x8786B1E> #<CL-SOURCE-FILE "modpackage" #x878675E> #<MODULE "documentation" #x878637E> #<CL-SOURCE-FILE "string" #x87857FE> #<CL-SOURCE-FILE "conditions" #x87853F6> #<MODULE "clos" #x8784FCE> #<MODULE "test" #x87847EE> #<CL-SOURCE-FILE "date" #x8783C56> #<CL-SOURCE-FILE "list" #x87837EE>) ASDF::IF-COMPONENT-DEP-FAILS: :FAIL ASDF::DEFAULT-COMPONENT-CLASS: NIL ASDF::DESCRIPTION: #<Unbound> ASDF::LONG-DESCRIPTION: #<Unbound> ASDF::AUTHOR: #<Unbound> ASDF::MAINTAINER: #<Unbound> ASDF::LICENCE: #<Unbound> ASDF:SOURCE-FILE: #P"/Development/Source/dev/Library/de/setf/utility/ utility.asd" 1 > (describe (first (asdf::module-components (rest (asdf::system- registered-p :de.setf.utility))))) #<CL-SOURCE-FILE "package" #x8786E86> Class: #<STANDARD-CLASS ASDF:CL-SOURCE-FILE> Wrapper: #<CCL::CLASS-WRAPPER ASDF:CL-SOURCE-FILE #x8643A8E> Instance slots ASDF::NAME: "package" ASDF:VERSION: #<Unbound> ASDF::IN-ORDER-TO: NIL ASDF::DO-FIRST: ((ASDF:COMPILE-OP (ASDF:LOAD-OP))) ASDF::INLINE-METHODS: NIL ASDF::PARENT: #<SYSTEM "de.setf.utility" #x878732E> ASDF::RELATIVE-PATHNAME: NIL ASDF::OPERATION-TIMES: #<HASH-TABLE :TEST EQL size 0/60 #x8786B86> ASDF::PROPERTIES: NIL 1 > (describe (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) #P"LIBRARY:package.lisp.newest" Type: LOGICAL-PATHNAME Class: #<BUILT-IN-CLASS LOGICAL-PATHNAME> TYPE: (LOGICAL-PATHNAME . #<CCL::CLASS-WRAPPER LOGICAL-PATHNAME #x809E75E>) 1: (:ABSOLUTE) 2: "package" 3: "lisp" %LOGICAL-PATHNAME-HOST: "LIBRARY" %LOGICAL-PATHNAME-VERSION: :NEWEST 1 > (in-package :asdf) #<Package "ASDF"> 1 > (defmethod component-relative-pathname ((component source-file)) (let ((*default-pathname-defaults* (make-pathname :name nil :type nil :directory nil :host nil :defaults (component-pathname (component- parent component))))) (merge-component-relative-pathname (slot-value component 'relative-pathname) (component-name component) (source-file-type component (component-system component))))) #<STANDARD-METHOD COMPONENT-RELATIVE-PATHNAME (SOURCE-FILE)> 1 > (describe (asdf::component-relative-pathname (first (asdf::module- components (rest (asdf::system-registered-p :de.setf.utility)))))) #P"package.lisp" Type: PATHNAME Class: #<BUILT-IN-CLASS PATHNAME> TYPE: (PATHNAME . #<CCL::CLASS-WRAPPER PATHNAME #x80A0F16>) %PATHNAME-DIRECTORY: (:RELATIVE) %PATHNAME-NAME: "package" %PATHNAME-TYPE: "lisp" %PHYSICAL-PATHNAME-VERSION: :NEWEST %PHYSICAL-PATHNAME-DEVICE: NIL 1 >

3 21

[asdf-devel] Q on compiling ASDF
by Daniel Herring 22 Feb '10

22 Feb '10

Apparently the ASDF-binary-locations (now asdf-output-locations) code originated from Slime. In Slime, this code allowed a simple loader to put incompatible fasls in separate paths. Now that ABL (AOL) is a standard part of ASDF, what is the proper way to bootstrap/write a (load (compile-file "asdf.lisp")) and have the fasls appear in implementation-specific directories? I ask this in relation to Nick's problem on Lispworks. Though I suspect a few extra stack frames (close to the patch I reported for clisp) are to blame for his immediate problem, in principle it would be good to always load the compiled form. Thanks, Daniel

3 2

[asdf-devel] Module-depends patch is in place
by Robert Goldman 13 Feb '10

13 Feb '10

OK, I have merged the module-depends branch into master and pushed the result. I suppose we should probably kill the module-depends branch on the public repo now.... Have a good weekend, all! r

2 2

[asdf-devel] minor suggestions
by Tobias C. Rittweiler 13 Feb '10

13 Feb '10

* How about adding a TEST-OP to ASDF's asd? * (asdf::version-satisfies (asdf:asdf-version) (asdf:asdf-version)) may be a trivial test case for version-satisfies. It's not obvious how to add test cases. -T.

3 2

[asdf-devel] Trying again
by Robert Goldman 12 Feb '10

12 Feb '10

OK, I have just pushed an alternative solution to the module dependency bug, this one (I believe) only triggered by INTRA-system dependencies. This one is also on the module-depends branch. This was achieved, as I said, at the cost of some hair, but I believe it does the right thing to the extent that I understand the right thing, and thinking about James's email convinced me that I /don't/ understand what is the right thing with respect to inter-system dependencies. Somewhat disappointingly, this commit agrees with the last one on the tests, which indicates that the tests need to be augmented... Additional reviews very welcome as are test results! [Fare: is your XCVB going to handle the kinds of inter-system issues I raised in my earlier email?] Feeling like Richard III, I can only say "A spec! A spec! My kingdom for a spec!" Cheers, r

3 14

[asdf-devel] Module dependencies still arguably wrong
by Robert Goldman 11 Feb '10

11 Feb '10

Earlier I claimed that the new ASDF TRAVERSE computed a conservative approximation to what needed to be recompiled. In particular, if we have (defsystem X :depends-on (Y) ....) With the new patch, if we do load-op on X, and Y has changed, X will be recompiled. In Classic ASDF, this would not happen. However, even with the new patch, this does not happen correctly. In particular, this case will not work properly (defsystem X :depends-on (Y) ...) (defsystem Z :depends-on (Y) ...) Let's imagine I load X and Z, which causes me to load Y. Now I modify some of the code of Y and reload X. ASDF will notice the change in Y and trigger a recompilation of X. BUT if I now (asdf:load-system :z), when we check the dependencies of Z and find Y, there will be nothing that needs to be done for Y, so recompilation of Z will /not/ be triggered. So the problem with trying to do a conservative estimation of what needs to be recompiled is that we don't store with a system object information about the state of things upon which it depends, so we cannot detect a change in the state of those dependencies..... Information about the last compilation of a system /is/ available in ASDF, but we don't store with X and Z information about the state of Z when they were compiled. To open YA can of worms, we also don't cache information about systems *across lisp sessions*. But I think I'll just raise that issue and then drop it like a hot potato. Cheers, r

2 2

[asdf-devel] asdf.x [was : Launchpad bug 502946
by james anderson 11 Feb '10

11 Feb '10

good morning; as is evident from the message attached below, for the past several days i have thought about things related to bug-502946, and asdf in general. my state-of-mind as of last week was, for the reasons alluded to below, that it is not productive to try to reason about system construction issues in the concrete context of asdf, -1.502, as it was at the time. i do not refer to with matters of configuration, or pathname mapping, or even 'elegant' bootstrapping, but rather to the need for a platform on which to think about and implement tools to - manage systems with articulated interfaces - manipulate system definitions programmatically - automate system definition construction and maintenance the larger concerns for integration with production, build, and deployment systems do all matter, just not right now. if one would set out the tasks in more detail, one could think about how to approach the issues, but at present, it is difficult to think even just about how asdf works. the only reasonable approach was to take asdf apart, isolate the functional components and reassemble them in a manner which allows to think about how a tool to manage program components works and how it might work. the current state is captured in de.setf.asdf.x[1]. there is a readme[2], which describes the intent, and the asdf-x.lisp[3] file, which reimplements asdf.lisp. there is also a unit test[4] which captures one interpretation of the 'strongly-dependent' requirement relation. on one hand, as a radical rational reconstruction of the original, it does not contribute to the main stream. on the other, the result provides the terms to think about what "intra-system" dependency is and a concrete context in which to implement projective proposals. cf. the commit[5] to support strong dependency by adding a state variable to the constraint machinery and adding the declaration to control it to the system definition language. most of which took the form of additional methods. --- [1]: http://github.com/lisp/de.setf.asdf.x [2]: http://github.com/lisp/de.setf.asdf.x/blob/master/README.md [3]: http://github.com/lisp/de.setf.asdf.x/blob/master/asdf-x.lisp [4]: http://github.com/lisp/de.setf.asdf.x/blob/master/test/launchpad/ bug-502946.lisp [5]: http://github.com/lisp/de.setf.asdf.x/commit/ 285a38e4ee18e10b5ce887d8ab2dd5e2d352aa0a Begin forwarded message: > From: james anderson <james.anderson(a)setf.de> > Date: 2010-02-07 23:45:56GMT+01:00 > To: rpgoldman(a)sift.info > Subject: Re: [asdf-devel] Launchpad bug 502946 > > good evening; > > you may, or may not be expecting this, but i got tired of trying to > understand traverse. > and tired of trying to understand parse-component, and defsystem, > and pathname resolution, and ... > i am now about 3/4 through rewriting asdf with clear protocols for > - definition operators (at all three levels - system, module, element) > - perform - with distinct qualifiers for requirement (dependency, > constituency) traversal, restart > - requirement and constituency specification canonicalization > - status evaluation for completion, features, other arbitrary > predicates > - settings/environment assertions per component > - pathname resolution which does not depend on the quirks of > *default-pathname-defaults* > > it's about the same amount of code - even though many of the > internal operators are decomposed to make the mechanism explicit. > > the most unsettling part is to ensure that - between the bnf in the > docs and the logic of the code, i've gotten the dependency > definitions right, but it won't be long. i've been chewing through > this since friday, so maybe by the time you're up tomorrow, there > will be some code up in github which passes the tests. the intent > is plug-compatible for the things i can understand, and i'm putting > it in a new package, so with a couple package-name swaps, one can > try things side-by-side. i will let you know. > > > On 2010-02-07, at 22:40 , Robert Goldman wrote: > >> Wrt https://bugs.launchpad.net/asdf/+bug/502946, >> module is not recompiled if _intra_-system dependency changes, I have >> spent quite some time groveling over the source code to TRAVERSE, >> and I >> believe that this is going to be quite difficult to fix. There are a >> number of assumptions baked into TRAVERSE, and some complicated >> uses of >> a variable scoped over the body of TRAVERSE, together with a >> number of >> local functions and a soupcon of unstated assumptions. >> >> I'm pretty sure that modules are substantially broken and, in the >> short >> term, I would discourage ASDF system designers from using them at >> all. >> >> I have been banging my head on this for at least a working day's >> worth >> of hours, and I need someone with whom to discuss the code, either by >> voice, IM or IRC. Email is not going to be high enough bandwidth. >> >> If anyone is interested in this enough to share the pain with me, >> please >> drop me a line and we can coordinate. >> >> Thanks, >> r >> >> _______________________________________________ >> asdf-devel mailing list >> asdf-devel(a)common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel >

1 0