asdf-devel

asdf-devel@common-lisp.net

5 participants
1375 discussions

Re: Wildcard modules support
by Ricardo 27 Jun '16

27 Jun '16

The intended purpose of wild-module as defined in asdf-contrib is to specify a module composed of the set of files that match a glob pattern. For example: (defsystem "my-system" ... :components ((:file ...) ... (:wild-module "plugins" :pathname "plugins/*.lisp"))) something like that, although the pathname specifier should rather be constructed in a platform-independent way. The usefulness of this feature is that it avoids having to perform manual maintenance on the system definition file every time a "plugin" is added or removed from the system. This is a pretty common thing on many projects and is easily handled by most (if not all) the system building tools I've come across in other platforms/languages. However, I put aside half a day yesterday to try to hack a similar thing into asdf and I found the project code in a less than ideal state with regards to documentation, clarity and overall structure and methodology, which surprised me because asdf3 was introduced as a rewrite in the hopes of achieving better extensibility, consistency and configurability. It's not clear at all where should someone plug these kind of features in or how to merge the behavior of a new specific component (not just a source-file subclass) into the whole control flow. I'd like to contribute but I can't spend any more evenings on this when hardly anyone seems to need it. For my own purposes, the solution proposed in http://stackoverflow.com/questions/24171268/how-to-make-defsystem-use-every… works fairly well (and, actually, something as simple as this might as well be a built-in feature). Sorry. Take care, rcn > I am, and I'm open to including this, but as I said before, not in its > current state of non-documentation. > > If we can get a description of its intended purpose, suitable for the > manual and to check its implementation, I will review it for inclusion. > > But ASDF has really grown, and I'm not in a position to incorporate any > more bits that aren't clearly documented. Rather the contrary: I'm > interested in better understanding and better documenting. > > Best, > r

3 2

Re: Wildcard modules support
by rcn 26 Jun '16

26 Jun '16

But loading all the required files from an "all.lisp" file makes asdf unaware of the existence of those files, doesn't it? I can use that as a workaround but it has several disadvantages: The relationships between those files and the rest of the system will be lost, and any changes with regards to compilation/loading configuration (bundling, loading from pre-compiled files) will have to be done manually in all.lisp, hidden from the project .asd file. Given that a workaround is necessary for this, using a reader macro in the system definition to unfold a list of :file components from a glob pattern seems like a better solution, but it's a shame that such a basic feature isn't supported out of the box. In my opinion, if you really want to push asdf as the standard system definition for Common Lisp, these kind of things (what the users need, after all) should be given more attention. It's unusual for a programming ecosytem to have a powerful system definition tool and still having to rely on Makefiles and roll-your-own hacks for some things. Don't get me wrong, I think asdf is a great tool and I thank you all for the effort you put on it. Kind regards, rcn >wild-modules were an example extension and/or contrib from the asdf 1 >days. There is a test for them in test/ but otherwise I don't believe >anyone uses them. If you have the asdf git installed, you can use them >with package-inferred-system. > >I believe that package-inferred-system is a better way to achieve a >similar effect, that allows for dependencies between sub-systems, >unlike wild-modules. I suppose a wildcard option to load all files in >a directory while using package-inferred-system dependencies would be >a nice feature to add. But so far, I found it more practical to have >an explicit file named "all.lisp" that includes all the files that >matter.

3 2

Wildcard modules support
by rcn 24 Jun '16

24 Jun '16

Hi all, Will wild-module support ever be fully and "officially" supported? Currently there's what appears to be a sample implementation as part of asdf/contrib but, if I recall correctly, asdf/contrib is bound to disappear eventually (actually, the documentation says this is an empty package). However, it's not clear how to use it, it's not documented and it's not clear if it will ever be a part of asdf or not. In my mind, this is an useful feature. Not for small systems (which, sadly, are the majority of Common Lisp open source projects), but for large systems where new files may be added to a directory to incorporate additional functionalities (a plug-in kind of thing). In these scenarios, wild-module definitions would ease development and system maintenance greatly. As far as I know, asdf is perfectly capable of implementing it and it looks to me like an implementation-agnostic feature, so I see no reason not to include as a core component. Thanks, rcn

3 2

Introspection of system version
by Frank 17 Jun '16

17 Jun '16

Hello, In the system definition we can specify a version; for example: (asdf:defsystem :foo :description "foo utility" :license "LGPL" :version (:read-file-form "VERSION") :serial t :components ((:file "foo"))) For example I can query the system license like this: (asdf:system-licence (asdf:find-system "foo")) How can I query the version of system foo at run time? Thanks, Frank

3 3

Fixed performance bug in UIOP:DIRECTORY-FILES
by Faré 14 Jun '16

14 Jun '16

I bumped ASDF to 3.1.7.1 after I fixed a performance bug in directory-files that made it quadratic instead of linear, due to some remove-duplicates used for logical-pathnames. The fix was to only use this quadratic behavior for LPNs. If you use LPNs, you deserve to lose on large directories. LPNs are for legacy Lisp code with relatively few files, only. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The best of all regimes is one where I am the head honcho of a worldwide empire. — Winston Churchill (his most famous quip, as translated to plain English)

3 6

Altering ASDF execution plan
by Vsevolod Dyomkin 30 May '16

30 May '16

Hi, I have a few questions about controlling ASDF execution of compile/load operations: 1. Suppose I have performed successful `load-op` on a certain system. After some time and no changes to the source code of the system nor its dependencies, I run `load-system` again on the same system. How can I ensure that no compile/load action is performed? 2. Suppose I have a system `foo` that depends on `bar`. Now, I explicitly call `(load-system :bar)`. How can I ensure that when I call `(load-system :foo)` no compile/load action is performed on `bar`? Sorry, maybe, these questions are trivial, but I couldn't find definite answers in the docs. --- Vsevolod Dyomkin +38-096-111-41-56 skype, twitter: vseloved

3 2

:FEATURE dependency-def
by Robert Goldman 20 May '16

20 May '16

Is it just me, or is the :FEATURE dependency-def form undocumented in the manual? It is *mentioned* in the grammar, but I don't see it documented -- except for the misleading mention that the :FEATURE *requirement* has been removed from ASDF. # This is used in :depends-on, as opposed to ``dependency,'' # which is used in :in-order-to dependency-def := simple-component-name | ( :feature FEATURE-EXPRESSION dependency-def ) | ( :version simple-component-name version-specifier ) | ( :require module-name ) I'm actually not sure what the semantics of this should be: 1. if the FEATURE is present, include dependency-def and 2a. If the FEATURE is absent FAIL or 2b. If the FEATURE is absent, quietly succeed. If the semantics is intended to be the latter -- and that's how I read the relevant code in find-component.lisp -- how does a programmer say "if this feature is not present, my system should not compile"? I stubbed my toe on this absent-mindedly compiling Didier's decLt system on Allegro Common Lisp. That system doesn't, and cannot, function on ACL, but ASDF tells us it has compiled successfully. Instead, what seems to happen is that the net.didierverna.declt.core system depends on (:feature :sbcl :sb-introspect) which simply seems to fail quietly on ACL, in what looks like the worst possible way: the attempt to laod net.didierverna.declt returns T, the guts of the system is not loaded (seems like this feature system somehow keeps core from continuing to load), and no error message is issued. Actually, I don't claim to understand why this happens -- it looks to me like RESOLVE-DEPENDENCY-SPEC should return NIL, and the load should continue (trying to load the :net.didierverna.declt.core system and succeeding -- or more likely choking when it hits a reference to :SB-INTROSPECT). But somehow it's failing to load. I'll see if I can make a minimal test case and file a bug, but I could use some guidance about what was intended. If the semantics of :FEATURE are like those of :IF-FEATURE, as I suspect, it seems like they aren't working here. And, if that's what's intended, how do we get the "fail if this feture is absent" behavior that a programmer might like? thanks, r

5 12

Re: How to fix arnesi:clean-op.
by Robert Goldman 15 May '16

15 May '16

I am inclined to agree with Andreas. I don't see that CLEAN-OP is impossible for ASDF. I can see that there are some issues that require thought, and for some systems additional coding by the developer, but I don't see that as an insuperable barrier. After all, "make clean" is in some sense impossible (it's generally hand-coded by the programmer), but we use it every day. I accept Faré's points about there being challenges to write an ideal CLEAN-OP. But I don't agree that this means that we can't write a good enough CLEAN-OP. After all, make manages "make clean." Yes, it requires some effort from the programmer, but who cares? Is that a reason we have to throw up our hands and refuse to do anything? Here are some specific responses to particular issues. On the other hand, while I don't agree that these make the CLEAN-OP impossible, I *do* agree that they are difficult issues. So... I'd be happy to see CLEAN-OP added, and I'd be happy to support this effort, *BUT* CLEAN-OP patches must be preceded -- by an informal specification that describes what an implementation should do, covering these cases. Now that ASDF is on everyone's critical path, we need to think a little before changing it, and not just bash out a hunk of code. Specific responses to Faré's issues: 1. clean-op is not defined in general. I believe that this means "do we clean the system" or "do we clean the operation" [as in arnesi, whose CLEAN-OP has a FOR-OP]? This seems similar to the notion that Faré already had, of providing a BUILD-OP that does "the default build action" for a system (e.g., maybe it builds the library? Maybe it does LOAD-OP? Maybe it does LOAD-OP and DOC-OP?). So one solution would be to have an UNMAKE-OP that does the inverse of the build-op, and then have a CLEAN-OP that undoes specific operations (like the arnesi CLEAN-OP). Then the UNMAKE-OP might default to being the CLEAN-OP for COMPILE-OP, which would handle probably 90% of systems. On the other hand, I'm pretty unhappy with the notion of the BUILD-OP, because it requires retraining every single user of ASDF to use BUILD-OP instead of LOAD-OP. That would be an enormous user interface fail. So this is not a step to be taken lightly. 2. I think we can simply make clean-op clean for the current implementation. 3. I believe we should mirror the semantics of :FORCE T and :FORCE ALL when we unmake. I.e., unbuild the dependencies only with :FORCE :ALL. :FORCE T would actually be a no-op. 4. "How do you deal with defsystem-depends-on and other dependencies not explicitly included in the plan?" DEFSYSTEM-DEPENDS-ON is already broken, and will be deprecated, except as a checkable declaration (see earlier postings on this list for a description of the issues). For the rest, I'd say same as above -- if there's something that won't appear in the build plan that one needs to clean, then the programmer must write an :IN-ORDER-TO for the CLEAN-OP so that ASDF can know what it needs to do. 5. "Are you sure there aren't bugs and omissions in asdf or any extension whereby some side output file is omitted from the list?" ow does make deal with a file that isn't listed in the rule for "make clean"? You miss it. Very sorry. Fix your system definition. Use (:in-order-to (clean-op ....)) 6. "What about randomly-named temporary files created during an operation that has been interrupted?" Maybe we are doing the Wrong Thing with these. If they are temporary files, maybe it's the wrong thing to put them in the cache and rename them? Maybe they should live in /tmp (or equivalent) and be moved from there? Then it's not our problem to fix anymore -- that's for tmpwatch. If you add your own rules that make temporary files, you have to add your own rules that remove them. 7. "What if load-op is not the only operation that matters, but e.g. doc-op generates LaTeX and PDF outputs but fails to be tracked by clean-op? Are you going to use the CLOS MOP to generate a list of all possible operations? What if that other operation isn't loaded in the current image? What if operation options cause different files to be generated? Or features? What about all secondary systems? Are you going to generate a plan for them, too?" See the answer to #1: ASDF is not going to solve this problem. Either the programmer solves it using definitions of PERFORM and IN-ORDER-TO, or it's not solved. So sad. But I don't see any reason to avoid the 90% solution because there isn't a 100% solution. Best, r

3 7

Error loading SBCL contribs
by Stelian Ionescu 08 May '16

08 May '16

I think SBCL commit 25c1769e30ff7404b52a9be663626d87f8ff75bc broke the use of deferred warnings. If I compile SBCL and exec ./run-sbcl.sh, I get this error during Quicklisp initialization: ; file: /home/hechee/lisp/quicklisp/quicklisp/impl.lisp ; in: DEFINE-IMPLEMENTATION-PACKAGE :SBCL ; (REQUIRE 'QL-IMPL::SB-POSIX) ; ; caught WARNING: ; completing compilation for #<REQUIRE-SYSTEM "sb-posix"> completed without its output file #P"/home/hechee/work/src/sbcl/obj/sbcl-home/contrib/sb-posix.build-report" ; ; caught WARNING: ; completing load for #<REQUIRE-SYSTEM "sb-posix"> completed without its input file #P"/home/hechee/work/src/sbcl/obj/sbcl-home/contrib/sb-posix.build-report" ; (REQUIRE 'QL-IMPL::SB-BSD-SOCKETS) ; ; caught WARNING: ; completing compilation for #<REQUIRE-SYSTEM "sb-bsd-sockets"> completed without its output file #P"/home/hechee/work/src/sbcl/obj/sbcl-home/contrib/sb-bsd-sockets.build-report" ; ; caught WARNING: ; completing load for #<REQUIRE-SYSTEM "sb-bsd-sockets"> completed without its input file #P"/home/hechee/work/src/sbcl/obj/sbcl-home/contrib/sb-bsd-sockets.build-report" While evaluating the form starting at line 129, column 0 of #P"/home/hechee/lisp/quicklisp/setup.lisp": While evaluating the form starting at line 57, column 0 of #P"/home/hechee/.cl-init/asdf-init.lisp": debugger invoked on a UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<THREAD "main thread" RUNNING {1002FC6A13}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "quicklisp" "impl"> Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [RETRY ] Retry compiling #<CL-SOURCE-FILE "quicklisp" "impl">. 1: [ACCEPT ] Continue, treating compiling #<CL-SOURCE-FILE "quicklisp" "impl"> as having been successful. 2: Retry ASDF operation. 3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration. 4: [RETRY ] Retry EVAL of current toplevel form. 5: [CONTINUE ] Ignore error and continue loading file "/home/hechee/lisp/quicklisp/setup.lisp". 6: [ABORT ] Abort loading file "/home/hechee/lisp/quicklisp/setup.lisp". 7: Retry EVAL of current toplevel form. 8: Ignore error and continue loading file "/home/hechee/.cl-init/asdf-init.lisp". 9: Abort loading file "/home/hechee/.cl-init/asdf-init.lisp". 10: Retry EVAL of current toplevel form. 11: Ignore error and continue userinit file "/home/hechee/.sbclrc". 12: Abort userinit file "/home/hechee/.sbclrc". 13: Skip to toplevel READ/EVAL/PRINT loop. 14: [EXIT ] Exit SBCL (calling #'EXIT, killing the process). (UIOP/LISP-BUILD:CHECK-LISP-COMPILE-RESULTS NIL T T "~/asdf-action::format-action/" ((#<ASDF/LISP-ACTION:COMPILE-OP > . #<ASDF/LISP-ACTION:CL-SOURCE-FILE "quicklisp" "impl">))) source: (ERROR 'COMPILE-FILE-ERROR :CONTEXT-FORMAT CONTEXT-FORMAT :CONTEXT-ARGUMENTS CONTEXT-ARGUMENTS) There's no obj/sbcl-home/contrib/sb-posix.build-report, or any build -report file, just several test-passed.test-report under obj/asdf -cache/ Funny though that I don't get this error if I exec sbcl through an absolute path, e.g. ~/work/src/sbcl/run-sbcl.sh, it's only when I call it from the source directory as ./run-sbcl.sh -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur.

1 0

How to fix arnesi:clean-op.
by TANIGUCHI Masaya 08 May '16

08 May '16

I want to do "make clean" on ASDF. I tried to use clean-op of ARNESI. I got a following message, WARNING: No dependency propagating scheme specified for operation class IT.BESE.ARNESI:CLEAN-OP. The class needs to be updated for ASDF 3.1 and specify appropriate propagation mixins. #<IT.BESE.ARNESI:CLEAN-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {1003A65533}> Definition of arnesi:clean-op is (defclass clean-op (asdf:operation) ((for-op :accessor for-op :initarg :for-op :initform 'asdf:compile-op)) (:documentation "Removes any files generated by an asdf component.")) (defmethod asdf:perform ((op clean-op) (c asdf:component)) "Delete all the output files generated by the component C." (dolist (f (asdf:output-files (make-instance (for-op op)) c)) (when (probe-file f) (delete-file f)))) (defmethod asdf:operation-done-p ((op clean-op) (c asdf:component)) "Returns T when the output-files of (for-op OP) C don't exist." (dolist (f (asdf:output-files (make-instance (for-op op)) c)) (when (probe-file f) (return-from asdf:operation-done-p nil))) t) How to fix it? Thanks. ta2gch

6 8

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

asdf-devel