- asdf-devel - mailman3.common-lisp.net

Wrong release version in front page
by Nikolaos Chatzikonstantinou 18 Nov '22

18 Nov '22

Hello, I noticed that the front page in <https://asdf.common-lisp.dev/> claims that the latest version is 3.3.5 but the release tarball <https://common-lisp.net/project/asdf/archives/asdf.tar.gz> unpacks to asdf-3.3.6. Also, do you sign the releases? Regards, Nikolaos Chatzikonstantinou

2 2

Change to allow loading linked asd files
by Nibby Nebbulous 17 Nov '22

17 Nov '22

Hi, I wonder if it possible that you would accept this change which allows finding of a system by a symbolicly linked ".asd" file. I've been running with this patch for about 10 years without issue. I'm sorry for not having test cases and documentation, but it seems time passing won't help that. I know this is probably a pointless change for everyone besides me, being the only user of my software, so I quite understand if you don't want to add it. Anyway, many thanks for your years of work on perhaps the most used CL software ever. diff --git a/find-system.lisp b/find-system.lisp index 0980a2cc..1a226aba 100644 --- a/find-system.lisp +++ b/find-system.lisp @@ -263,11 +263,16 @@ PREVIOUS-PRIMARY when not null is the primary system for the PREVIOUS system." (locate-system name) (assert (eq foundp (and (or found-system pathname previous) t)))) (let ((previous-pathname (system-source-file previous)) - (system (or previous found-system))) + (system (or previous found-system)) + real-name) (when (and found-system (not previous)) (register-system found-system)) (when (and system pathname) (setf (system-source-file system) pathname)) + (when (and pathname + (setf real-name (pathname-name pathname)) + (not (equal real-name name))) + (setf name real-name)) (if-let ((stamp (get-file-stamp pathname))) (let ((up-to-date-p (and previous previous-primary

3 4

Handling of readme and license files
by Russell Sim 21 Oct '22

21 Oct '22

Hey, I'm working with a project that bundles up CL systems, and then copies them to a remote host. To avoid copying unnecessary files, we rely on the components and additional-input-files to find all the files needed to load the system on the remote host. I found some ASD files to load on the remote host. These systems are expecting to read a long-description, usually a readme file. All of these projects are using reader macros to load this data, and none of them declare the file as a :static-file component. It's also often the case that the license of a project requires a copy of the license be shipped with the project. Yet rarely do I see any project declaring a :static-file for one in their asd file. Am I correct in my understanding that using :static-files is this the current recommended way to declare these files? Is there any plan for ASDF to improve the handling of these 2 cases? I see other packaging systems often have a special field for the license file, and having a :read-file form for the long-descripton much like the version field has, would improve handling of these readme's greatly. At the moment asd authors don't have an easy way to test asd evaluation when files like the readme aren't present. Leading to error prone code like this. :long-description #.(uiop:read-file-string (uiop:subpathname *load-pathname* "README.md")) I pulled a sample of 800 packages from quicklisp, 95 of them are using either uiop:read-file-string (65) or a safer, more verbose with-open-file (30) form. Thank you for your work maintaining this essential system. Regards, Russell Sim

4 12

ASDF 3.3.6 is released
by Robert Goldman 07 Aug '22

07 Aug '22

Today we release ASDF 3.3.6, the sixth bugfix release for the 3.3 release series, and -- it is devoutly to be hoped -- the final one. We urge implementations that are currently bundling previous versions of ASDF -- and especially those bundling 3.3.0 through 3.3.5 -- to upgrade to 3.3.6 at their earliest convenience. As maintainer, my primary role continues to be coordination and some limited quality control. The real work for this release has been done by many contributors and bug-reporters, to whom the Common Lisp community owes a tremendous vote of thanks. Their names are listed on the Changelog, appended below. An extra special thanks is due to my co-maintainer, Eric Timmons, who has provided many fixes and also has overhauled and much improved our regression testing. We *hope* that the next release will be 3.4, introducing new functionality but which, as the numbering suggests, will be backward compatible. Best, Robert # Changelog * Fix bug loading ASDF system definitions from jar files on ABCL. See !190 * Fix bug with version numbering after reload. See !194 and #94. Thanks to Eric Timmons. * Fix bug in return value of UPGRADE-ASDF. See !195. Again, thanks to Eric Timmons. * Multiple improvements to documentation and tests. Thanks to Nathan Ringo and Phoebe Goldman. * Add clasp support for uiop/launch program. Thanks to Christian Schafmeister. * Miscellaneous minor bug fixes and tweaks. * Don't let *package* binding leak into ASDF compile operations. Thanks to Phoebe Goldman.

1 0

Closed the last of the issues on the 3.3.6 milestone
by Robert Goldman 02 Aug '22

02 Aug '22

So I am definitely moving towards a release real soon now. So please test the new ASDF as you can, and report any issues you see.

1 0

Trying once more to get 3.3.6 out the door
by Robert Goldman 27 Jul '22

27 Jul '22

I have poked all the merge requests that were tagged for 3.3.6 If these are close to working, I will get them merged, otherwise I will drop them from the milestone, with an eye to getting 3.3.6 out soon. If you get a chance to test your systems against the current version 3.3.5.10 looking for any new problems, I would be very grateful. The regression tests can take us only so far.

1 0

[RFC] subsystems vs. modules
by Didier Verna 18 May '22

18 May '22

Hi there, until now, I had a tendency to split a library into several systems (more or less without thinking much) as soon as such or such feature was conditional (e.g. depending on CFFI availability, thread support, etc.). Now I realize that in many cases, a conditionally included module or component would suffice, and so I'm starting to think that maybe I should do that, unless it makes sense for the conditional part to be loaded sort of standalone, independently from the rest, in which case a subsystem is more appropriate. So my question is: WDYT, is there a general recommendation about this? Thanks. -- Resistance is futile. You will be jazzimilated. Lisp, Jazz, Aïkido: http://www.didierverna.info

4 4

Re: Define a Simple Echo-Op
by zacque 06 May '22

06 May '22

Hi phoebe, > [[PGP Signed Part:Undecided]] > I'll still get the warnings (even with one new warning: "No dependency > propagating scheme specified..."): > > Perhaps it was imprecise of me to say that defining a method on COMPONENT-DEPENDS-ON will prevent the deprecation warning. > Causing the default method to be invoked on an OPERATION that is not a direction-OPERATION is deprecated; defining a more > specific method which does CALL-NEXT-METHOD still invokes the deprecated behavior. > > What I meant was subclassing ASDF:OPERATION only > > You can subclass OPERATION without a direction-OPERATION, but you have to completely override COMPONENT-DEPENDS-ON > with your own behavior, never invoking the built-in method. I strongly recommend against doing this. > > So, all operations are by default downward and sideway unless they are > subclass of (OR DOWNWARD-OPERATION UPWARD-OPERATION SIDEWAY-OPERATION > SELFWARD-OPERATION NON-PROPAGATING-OPERATION) > > Er... I'm not comfortable with saying that. In ASDF 2, all operations behaved in a way that is now called being downward and > sideway. ASDF 3 treats operations which don't specify their dependency relationship in a way compatible with ASDF 2 to give > maintainers a chance to update old extensions. The beautiful dream is that, one day, the compatibility behavior will be replaced > with a hard error. Please do not treat the compatibility behavior as a part of ASDF 3's interface. > >From our exchange, I think its safe to summarise into this table: "Table of ASDF operation classes to subclass and the corresponding methods to define on your custom class." | | OPERATION | <direction>-OPERATION | *-op | |----------------------+------------+--------------------------+------| | perform | X | X | X | | component-depends-on | X (note 1) | | | | input-files | X | (Depending on your need) | | | output-files | X | (Depending on your need) | | Note 1: You have to completely override COMPONENT-DEPENDS-ON for your custom class without invoking (CALL-NEXT-METHOD). As a new ASDF extension writer, I think having a table like this helps a lot to easily getting started. > Is it a linear chain or a "network chain" of > operations? > > The latter. When you define a subclass of SELFWARD-OPERATION, you specify what other operation(s) it is selfward to by > overriding the :INITFORM of the slot SELFWARD-OPERATION. The value you provide may be either a single operation designator, > or a list of operation designators. For example, (LOAD-OP COMPONENT) depends on both (COMPILE-OP COMPONENT) and > (PREPARE-OP COMPONENT), so its SELFWARD-OPERATION slot is defined: > > (selfward-operation :initform '(prepare-op compile-op) :allocation :class) > > can be done in parallel, > > But how can I > express the operation dependencies such that ASDF knows PRINT-OP depends > on ECHO-OP? > > (defclass print-op (asdf:selfward-operation) > ((asdf:selfward-operation :initform 'echo-op :allocation :class))) > Thanks! I have tried it out and it works. Still need some time to understand its behaviour though. > ASDF as it exists now is not capable of running anything in parallel. Faré has an experimental project to do this, called POIU, but I > can't speak for its status. Until very recently, SBCL was incapable of compiling files in parallel, and still today, lots of code does > not-thread-safe stuff at load-time, so I'm not convinced it's practically possible to make ASDF parallel via shared-memory threaded > concurrency. I believe POIU compiles each component in a separate process, then loads them sequentially. I see, that's good to know. At least it's an implementation limitation and not a design limitation I guess. -- Regards, zacque

2 1

Re: Define a Simple Echo-Op
by zacque 28 Apr '22

28 Apr '22

Hi phoebe, Thanks for your thoughtful reply! > A few things. > > 1. ECHO-OP should not be selfward. SELFWARD-OPERATION is for (OPERATION COMPONENT) pairs which depend on > (DIFFERENT-OPERATION COMPONENT) for the same COMPONENT. For example, LOAD-OP is selfward with respect to > COMPILE-OP, because in order to perform (load-op FILE), you must first perform (compile-op FILE). Your ECHO-OP has no such > dependency. In this case, I believe you want ECHO-OP to be downward and sideways, meaning that (ECHO-OP MODULE) > depends on (ECHO-OP MODULE-COMPONENT) for each of the children MODULE-COMPONENTs of the MODULE, and that > (ECHO-OP SOURCE-FILE) depends on (ECHO-OP EARLIER-SOURCE-FILE) for each of the EARLIER-SOURCE-FILEs in > SOURCE-FILE's :DEPENDS-ON list. This way, when you call (OPERATE 'ECHO-OP (FIND-SYSTEM "whatever")), ASDF will do a > depth-first dependency-order traversal of your system. After playing a little with downward and sideway, I can observe that downward-operation provides a depth-search/"postorder" traversal for the current system; sideway-operation provides a dependency-first traversal for dependent systems. * Note: Use :force all with OPERATE to force reloading all dependent systems. E.g. (asdf:operate 'echo-op:echo-op :echo-op-test :force :all) * Question: Why can't I subclass ASDF:OPERATION? This is unexpected since ASDF:OPERATION is the base class for all operations. My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op) (defclass echo-op (asdf:operation) ()) (defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c))) ------------------------------------------------- Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> has input files: Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> has input files: #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {1015FB0BC3}> ------------------------------------------------- > 2. Your COMPONENT-DEPENDS-ON method is wrong. No pair of (OPERATION COMPONENT) should ever depend on the same > (OPERATION COMPONENT). What you're saying is, "in order to perform (ECHO-OP FILE), you must first perform (ECHO-OP > FILE)." > > 3. For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or > NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method. Why is my COMPONENT-DEPENDS-ON method wrong? It's not obvious that one doesn't need to define a COMPONENT-DEPENDS-ON method for >=1 *-OPERATION. Is it stated anywhere? I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on." and following the examples in cffi-grovel: https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/… That said, my code works fine by removing my COMPONENT-DEPENDS-ON method. And I can see what you meant by (ECHO-OP FILE) depending on itself by printing it out: My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op) (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ()) (defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c))) ------------------------------------------------- Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> depends on (#<ECHO-OP >) (#<ECHO-OP > #<CL-SOURCE-FILE "echo-op-test" "package"> #<CL-SOURCE-FILE "echo-op-test" "example"> #<CL-SOURCE-FILE "echo-op-test" "main">) (DEFINE-OP echo-op-test) #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {10162397C3}> ------------------------------------------------- But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes? > 4. Most (OPERATION COMPONENT) pairs have very uninteresting sets of input files. (COMPILE-OP CL-SOURCE-FILE) has one > input file, the .lisp source file. (LOAD-OP CL-SOURCE-FILE) has one input file, the .fasl compiled file. (ECHO-OP CL-SOURCE-FILE) > will have no input files at all, unless you define a method on INPUT-FILES to list them. Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP. Again, I thought ASDF is smart enough to infer it for me? To quote the manual: "A method for this function is often not needed, since ASDF has a pretty clever default input-files mechanism." Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does? > I think the following definition of ECHO-OP might be enlightening to you: > > (uiop:define-package :echo-op > (:use :cl) > (:export #:echo-op)) > (in-package :echo-op) > > (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ()) > > (defun print-input-files (op c) > (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" > op c (asdf:input-files op c))) > > (defun print-dependencies (op c) > (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" > op c (asdf:component-depends-on op c))) > > (defmethod asdf:perform ((op echo-op) c) > (flet ((do-operations (thunk) > (funcall thunk op c) > (funcall thunk (asdf:make-operation 'asdf:compile-op) c) > (funcall thunk (asdf:make-operation 'asdf:load-op) c))) > (format t "~&~%Input files for component ~a with a variety of operations:~%~%" c) > (do-operations #'print-input-files) > (format t "~&~%Dependencies for component ~a with a variety of operations:~%~%" c) > (do-operations #'print-dependencies))) > > Note that: > 1. The only method I have defined is on PERFORM, and it is a primary method, not an :AROUND method. ASDF already has all > the COMPONENT-DEPENDS-ON methods I need. > 2. I print the COMPONENT-DEPENDS-ON list in addition to the INPUT-FILES list. > 3. I print both the COMPONENT-DEPENDS-ON and INPUT-FILES lists for all three of ECHO-OP, COMPILE-OP and LOAD-OP. > > I recommend you load this version, try (ASDF:OPERATE 'ECHO-OP:ECHO-OP (ASDF:FIND-SYSTEM "echo-op-test") :FORCE T) and > see what output you get. Thanks! It's indeed very enlightening. And I've separated it out into few parts to play with, as shown in the code snippet I pasted above. One question: How do you know whether to define PERFORM primary method or an :AROUND method for the custom operation class? It's not obvious to me which one to choose for different use cases. Thanks! -- Regards, zacque

3 4

Re: Define a Simple Echo-Op
by Robert Goldman 27 Apr '22

27 Apr '22

On 27 Apr 2022, at 11:15, phoebe Goldman wrote: >> According to the docs, most operations inherit from self ward-operation, which would take the component-pathname as an input file (and might take others). > > I think input file sets are orthogonal to dependency direction. SELFWARD-OPERATION means that (OP COMPONENT) depends on (DIFFERENT-OP COMPONENT), like how (LOAD-OP LISP-SOURCE-FILE) depends on (COMPILE-OP LISP-SOURCE-FILE). That doesn't mean that (LOAD-OP LISP-SOURCE-FILE) takes COMPONENT-PATHNAME as an INPUT-FILE. This is what I see in the source, but I haven't had enough time to read it over carefully (or really, at all): ``` ;; An action with a selfward-operation by default gets its input-files from the output-files of ;; the actions using selfward-operations it depends on (and the same component), ;; or if there are none, on the component-pathname of the component if it's a file ;; -- and then on the results of the next-method. (defmethod input-files ((o selfward-operation) (c component)) `(,@(or (loop :for dep-o :in (ensure-list (selfward-operation o)) :append (or (output-files dep-o c) (input-files dep-o c))) (if-let ((pathname (component-pathname c))) (and (file-pathname-p pathname) (list pathname)))) ,@(call-next-method)))) ```

1 0