Hi phoebe,
Thanks for your thoughtful reply!
A few things.
- ECHO-OP should not be selfward. SELFWARD-OPERATION is for (OPERATION COMPONENT) pairs which depend on
(DIFFERENT-OPERATION COMPONENT) for the same COMPONENT. For example, LOAD-OP is selfward with respect to COMPILE-OP, because in order to perform (load-op FILE), you must first perform (compile-op FILE). Your ECHO-OP has no such dependency. In this case, I believe you want ECHO-OP to be downward and sideways, meaning that (ECHO-OP MODULE) depends on (ECHO-OP MODULE-COMPONENT) for each of the children MODULE-COMPONENTs of the MODULE, and that (ECHO-OP SOURCE-FILE) depends on (ECHO-OP EARLIER-SOURCE-FILE) for each of the EARLIER-SOURCE-FILEs in SOURCE-FILE's :DEPENDS-ON list. This way, when you call (OPERATE 'ECHO-OP (FIND-SYSTEM "whatever")), ASDF will do a depth-first dependency-order traversal of your system.
After playing a little with downward and sideway, I can observe that downward-operation provides a depth-search/"postorder" traversal for the current system; sideway-operation provides a dependency-first traversal for dependent systems.
* Note: Use :force all with OPERATE to force reloading all dependent systems. E.g. (asdf:operate 'echo-op:echo-op :echo-op-test :force :all)
* Question: Why can't I subclass ASDF:OPERATION? This is unexpected since ASDF:OPERATION is the base class for all operations.
My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op)
(defclass echo-op (asdf:operation) ())
(defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c))) -------------------------------------------------
Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> has input files: Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> has input files: #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {1015FB0BC3}> -------------------------------------------------
- Your COMPONENT-DEPENDS-ON method is wrong. No pair of (OPERATION COMPONENT) should ever depend on the same
(OPERATION COMPONENT). What you're saying is, "in order to perform (ECHO-OP FILE), you must first perform (ECHO-OP FILE)."
- For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or
NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method.
Why is my COMPONENT-DEPENDS-ON method wrong? It's not obvious that one doesn't need to define a COMPONENT-DEPENDS-ON method for >=1 *-OPERATION. Is it stated anywhere?
I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on." and following the examples in cffi-grovel: https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/g...
That said, my code works fine by removing my COMPONENT-DEPENDS-ON method. And I can see what you meant by (ECHO-OP FILE) depending on itself by printing it out:
My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op)
(defclass echo-op (asdf:sideway-operation asdf:downward-operation) ())
(defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c))) -------------------------------------------------
Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> depends on (#<ECHO-OP >) (#<ECHO-OP > #<CL-SOURCE-FILE "echo-op-test" "package"> #<CL-SOURCE-FILE "echo-op-test" "example"> #<CL-SOURCE-FILE "echo-op-test" "main">) (DEFINE-OP echo-op-test) #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {10162397C3}> -------------------------------------------------
But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes?
- Most (OPERATION COMPONENT) pairs have very uninteresting sets of input files. (COMPILE-OP CL-SOURCE-FILE) has one
input file, the .lisp source file. (LOAD-OP CL-SOURCE-FILE) has one input file, the .fasl compiled file. (ECHO-OP CL-SOURCE-FILE) will have no input files at all, unless you define a method on INPUT-FILES to list them.
Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP.
Again, I thought ASDF is smart enough to infer it for me? To quote the manual:
"A method for this function is often not needed, since ASDF has a pretty clever default input-files mechanism."
Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does?
I think the following definition of ECHO-OP might be enlightening to you:
(uiop:define-package :echo-op (:use :cl) (:export #:echo-op)) (in-package :echo-op)
(defclass echo-op (asdf:sideway-operation asdf:downward-operation) ())
(defun print-input-files (op c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c)))
(defun print-dependencies (op c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c)))
(defmethod asdf:perform ((op echo-op) c) (flet ((do-operations (thunk) (funcall thunk op c) (funcall thunk (asdf:make-operation 'asdf:compile-op) c) (funcall thunk (asdf:make-operation 'asdf:load-op) c))) (format t "~&~%Input files for component ~a with a variety of operations:~%~%" c) (do-operations #'print-input-files) (format t "~&~%Dependencies for component ~a with a variety of operations:~%~%" c) (do-operations #'print-dependencies)))
Note that:
- The only method I have defined is on PERFORM, and it is a primary method, not an :AROUND method. ASDF already has all
the COMPONENT-DEPENDS-ON methods I need. 2. I print the COMPONENT-DEPENDS-ON list in addition to the INPUT-FILES list. 3. I print both the COMPONENT-DEPENDS-ON and INPUT-FILES lists for all three of ECHO-OP, COMPILE-OP and LOAD-OP.
I recommend you load this version, try (ASDF:OPERATE 'ECHO-OP:ECHO-OP (ASDF:FIND-SYSTEM "echo-op-test") :FORCE T) and see what output you get.
Thanks! It's indeed very enlightening. And I've separated it out into few parts to play with, as shown in the code snippet I pasted above.
One question: How do you know whether to define PERFORM primary method or an :AROUND method for the custom operation class? It's not obvious to me which one to choose for different use cases.
Thanks!
Phoebe Goldman writes (but can't post):
For some frustrating reason I can't email the asdf-devel list, so cc'ing rpgoldman & he can forward.
Why can't I subclass ASDF:OPERATION? This is unexpected since ASDF:OPERATION is the base class for all operations.
The various UPWARD- DOWNWARD- etc -OPERATION classes are subclasses of OPERATION. You can explicitly add OPERATION to your superclasses list alongside SIDEWAY-OPERATION if you want to, but it's redundant.
In ASDF 2, the direction-OPERATION classes didn't exist; you just subclassed OPERATION directly. All operations were implicitly DOWNWARD and SIDEWAY unless you wrote your own COMPONENT-DEPENDS-ON method. ASDF 3 added some built-in operations that were not SIDEWAY/DOWNWARD (PREPARE-OP is UPWARD, for example), so Faré codified all the sane dependency relationships as the direction-OPERATION classes, and deprecated the default COMPONENT-DEPENDS-ON method for OPERATION. In ASDF 3, if you subclass OPERATION directly and don't define a method on COMPONENT-DEPENDS-ON, you get a deprecation warning.
I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on."
This is wrong; I don't know why the manual says that. You still can define a method on COMPONENT-DEPENDS-ON, but there's no reason to unless you have a really weird operation. Writing your own COMPONENT-DEPENDS-ON methods is error-prone and a pain in the ass, so it's much easier to subclass one of the direction-DEPENDS-ON methods and get a built-in method that does the right thing.
But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes?
Honestly, I have no clue why there's an (ECHO-OP . NIL) pair at the head of the COMPONENT-DEPENDS-ON list for an ECHO-OP. But MAP-DIRECT-DEPENDENCIES ignores it. Shrug.
"A method for [INPUT-FILES] is often not needed, since ASDF has a pretty clever default input-files mechanism."
Again, I don't know why the manual says that. You don't need to define INPUT-FILES methods for built-in operations, but you will need to define them for custom operations that take files as input. Most user-defined operations subclass either COMPILE-OP or LOAD-OP, which already have their INPUT-FILES (and OUTPUT-FILES) methods, but I've had to define INPUT-FILES methods in the past.
Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does?
COMPONENT-PATHNAME.
Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP.
Taking the COMPONENT-PATHNAME as the sole INPUT-FILE can't be the default, because lots of operations do something different. If you think of PREPARE-OP, COMPILE-OP and LOAD-OP as being the "normal" operations on a LISP-SOURCE-FILE, only COMPILE-OP takes the source file as an INPUT-FILE. LOAD-OP takes as an INPUT-FILE the compiled fasl, which is the OUTPUT-FILES of COMPILE-OP. I don't think PREPARE-OP takes any INPUT-FILES at all, but I could be wrong.
One question: How do you know whether to define PERFORM primary method or an :AROUND method for the custom operation class? It's not obvious to me which one to choose for different use cases.
Write an :AROUND method if you're subclassing an existing operation like LOAD-OP to modify its behavior, and you intend to use CALL-NEXT-METHOD to invoke the default behavior for your superclass at some point in your method. This is useful if you want to invoke the next method in a dynamic context e.g. with handlers or restarts bound, or with local bindings of special variables; or if you want to conditionally invoke the next method sometimes but not always. Actually, you can do all of these things in a primary method via CALL-NEXT-METHOD, so now that I think of it, just define a primary method. Write an :AROUND method only if you're defining a mix-in class that other people will subclass which wants to do those things.
If you're subclassing an existing operation and you want to add additional behavior, but you still want the default behavior to run unconditionally in a default dynamic context, define a :BEFORE or :AFTER method as appropriate.
Write a primary method if you're defining a new method, or if you want to completely override the behavior of an existing operation. Or if you want to do the things I said earlier with CALL-NEXT-METHOD.
When in doubt, write a primary method.
cheers, phoebe
On Apr 27, 2022, at 3:21 AM, zacque technical+asdfdevel@zacque.tk wrote:
Hi phoebe,
Thanks for your thoughtful reply!
A few things.
- ECHO-OP should not be selfward. SELFWARD-OPERATION is for (OPERATION COMPONENT) pairs which depend on
(DIFFERENT-OPERATION COMPONENT) for the same COMPONENT. For example, LOAD-OP is selfward with respect to COMPILE-OP, because in order to perform (load-op FILE), you must first perform (compile-op FILE). Your ECHO-OP has no such dependency. In this case, I believe you want ECHO-OP to be downward and sideways, meaning that (ECHO-OP MODULE) depends on (ECHO-OP MODULE-COMPONENT) for each of the children MODULE-COMPONENTs of the MODULE, and that (ECHO-OP SOURCE-FILE) depends on (ECHO-OP EARLIER-SOURCE-FILE) for each of the EARLIER-SOURCE-FILEs in SOURCE-FILE's :DEPENDS-ON list. This way, when you call (OPERATE 'ECHO-OP (FIND-SYSTEM "whatever")), ASDF will do a depth-first dependency-order traversal of your system.
After playing a little with downward and sideway, I can observe that downward-operation provides a depth-search/"postorder" traversal for the current system; sideway-operation provides a dependency-first traversal for dependent systems.
- Note:
Use :force all with OPERATE to force reloading all dependent systems. E.g. (asdf:operate 'echo-op:echo-op :echo-op-test :force :all)
- Question:
Why can't I subclass ASDF:OPERATION? This is unexpected since ASDF:OPERATION is the base class for all operations.
My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op)
(defclass echo-op (asdf:operation) ())
(defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c)))
Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> has input files: Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> has input files: #<ECHO-OP:ECHO-OP >
#<ASDF/PLAN:SEQUENTIAL-PLAN {1015FB0BC3}>
- Your COMPONENT-DEPENDS-ON method is wrong. No pair of (OPERATION COMPONENT) should ever depend on the same
(OPERATION COMPONENT). What you're saying is, "in order to perform (ECHO-OP FILE), you must first perform (ECHO-OP FILE)."
- For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or
NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method.
Why is my COMPONENT-DEPENDS-ON method wrong? It's not obvious that one doesn't need to define a COMPONENT-DEPENDS-ON method for >=1 *-OPERATION. Is it stated anywhere?
I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on." and following the examples in cffi-grovel: https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/g... https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/grovel/asdf.lisp#L66
That said, my code works fine by removing my COMPONENT-DEPENDS-ON method. And I can see what you meant by (ECHO-OP FILE) depending on itself by printing it out:
My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op)
(defclass echo-op (asdf:sideway-operation asdf:downward-operation) ())
(defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c)))
Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> depends on (#<ECHO-OP >) (#<ECHO-OP > #<CL-SOURCE-FILE "echo-op-test" "package"> #<CL-SOURCE-FILE "echo-op-test" "example"> #<CL-SOURCE-FILE "echo-op-test" "main">) (DEFINE-OP echo-op-test) #<ECHO-OP:ECHO-OP >
#<ASDF/PLAN:SEQUENTIAL-PLAN {10162397C3}>
But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes?
- Most (OPERATION COMPONENT) pairs have very uninteresting sets of input files. (COMPILE-OP CL-SOURCE-FILE) has one
input file, the .lisp source file. (LOAD-OP CL-SOURCE-FILE) has one input file, the .fasl compiled file. (ECHO-OP CL-SOURCE-FILE) will have no input files at all, unless you define a method on INPUT-FILES to list them.
Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP.
Again, I thought ASDF is smart enough to infer it for me? To quote the manual:
"A method for this function is often not needed, since ASDF has a pretty clever default input-files mechanism."
Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does?
I think the following definition of ECHO-OP might be enlightening to you:
(uiop:define-package :echo-op (:use :cl) (:export #:echo-op)) (in-package :echo-op)
(defclass echo-op (asdf:sideway-operation asdf:downward-operation) ())
(defun print-input-files (op c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c)))
(defun print-dependencies (op c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c)))
(defmethod asdf:perform ((op echo-op) c) (flet ((do-operations (thunk) (funcall thunk op c) (funcall thunk (asdf:make-operation 'asdf:compile-op) c) (funcall thunk (asdf:make-operation 'asdf:load-op) c))) (format t "~&~%Input files for component ~a with a variety of operations:~%~%" c) (do-operations #'print-input-files) (format t "~&~%Dependencies for component ~a with a variety of operations:~%~%" c) (do-operations #'print-dependencies)))
Note that:
- The only method I have defined is on PERFORM, and it is a primary method, not an :AROUND method. ASDF already has all
the COMPONENT-DEPENDS-ON methods I need. 2. I print the COMPONENT-DEPENDS-ON list in addition to the INPUT-FILES list. 3. I print both the COMPONENT-DEPENDS-ON and INPUT-FILES lists for all three of ECHO-OP, COMPILE-OP and LOAD-OP.
I recommend you load this version, try (ASDF:OPERATE 'ECHO-OP:ECHO-OP (ASDF:FIND-SYSTEM "echo-op-test") :FORCE T) and see what output you get.
Thanks! It's indeed very enlightening. And I've separated it out into few parts to play with, as shown in the code snippet I pasted above.
One question: How do you know whether to define PERFORM primary method or an :AROUND method for the custom operation class? It's not obvious to me which one to choose for different use cases.
Thanks!
-- Regards, zacque
On 27 Apr 2022, at 10:26, Robert Goldman wrote:
Phoebe Goldman writes (but can't post):
I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on."
This is wrong; I don't know why the manual says that. You still can define a method on COMPONENT-DEPENDS-ON, but there's no reason to unless you have a really weird operation. Writing your own COMPONENT-DEPENDS-ON methods is error-prone and a pain in the ass, so it's much easier to subclass one of the direction-DEPENDS-ON methods and get a built-in method that does the right thing.
I will fix the manual RN.
But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes?
Honestly, I have no clue why there's an (ECHO-OP . NIL) pair at the head of the COMPONENT-DEPENDS-ON list for an ECHO-OP. But MAP-DIRECT-DEPENDENCIES ignores it. Shrug.
"A method for [INPUT-FILES] is often not needed, since ASDF has a pretty clever default input-files mechanism."
Again, I don't know why the manual says that. You don't need to define INPUT-FILES methods for built-in operations, but you will need to define them for custom operations that take files as input. Most user-defined operations subclass either COMPILE-OP or LOAD-OP, which already have their INPUT-FILES (and OUTPUT-FILES) methods, but I've had to define INPUT-FILES methods in the past.
I will update that, too. TBH, I'm not sure that this is up-to-date with Fare's refactoring that introduced the `additional-input-files` generic function, but I don't have enough time today to dig into this burger.
Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does?
COMPONENT-PATHNAME.
Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP.
Taking the COMPONENT-PATHNAME as the sole INPUT-FILE can't be the default, because lots of operations do something different. If you think of PREPARE-OP, COMPILE-OP and LOAD-OP as being the "normal" operations on a LISP-SOURCE-FILE, only COMPILE-OP takes the source file as an INPUT-FILE. LOAD-OP takes as an INPUT-FILE the compiled fasl, which is the OUTPUT-FILES of COMPILE-OP. I don't think PREPARE-OP takes any INPUT-FILES at all, but I could be wrong.
According to the docs, most operations inherit from `self ward-operation`, which would take the component-pathname as an input file (and might take others).
If this is wrong, please let me know and I will adjust the documentation.
Getting more of this discussion into the manual would be great, since there's lots of good guidance here.
A quick note to point everyone at the doc fixes: https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/207
(Ignore my previous email with the wrong "From:" sender field)
For some frustrating reason I can't email the asdf-devel list, so cc'ing rpgoldman & he can forward.
Noted, I'll reply to rpgoldman and cc you then (for better thread viewing?).
The various UPWARD- DOWNWARD- etc -OPERATION classes are subclasses of OPERATION. You can explicitly add OPERATION to your superclasses list alongside SIDEWAY-OPERATION if you want to, but it's redundant.
What I meant was subclassing ASDF:OPERATION only, as shown in the following code.
In ASDF 2, the direction-OPERATION classes didn't exist; you just subclassed OPERATION directly. All operations were implicitly DOWNWARD and SIDEWAY unless you wrote your own COMPONENT-DEPENDS-ON method. ASDF 3 added some built-in operations that were not SIDEWAY/DOWNWARD (PREPARE-OP is UPWARD, for example), so Faré codified all the sane dependency relationships as the direction-OPERATION classes, and deprecated the default COMPONENT-DEPENDS-ON method for OPERATION.
I see, thank you for your explanation.
In ASDF 3, if you subclass OPERATION directly and don't define a method on COMPONENT-DEPENDS-ON, you get a deprecation warning.
This is not true. For my ASDF version "3.3.5.7". If I do this: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op)
(defclass echo-op (asdf:operation) ())
(defmethod asdf:component-depends-on ((op echo-op) c) (call-next-method))
(defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c))) -------------------------------------------------
I'll still get the warnings (even with one new warning: "No dependency propagating scheme specified..."): ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) WARNING: No dependency propagating scheme specified for operation class ECHO-OP:ECHO-OP. The class needs to be updated for ASDF 3.1 and specify appropriate propagation mixins. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> depends on (DEFINE-OP echo-op-test) (#<ECHO-OP >) (#<ECHO-OP > #<CL-SOURCE-FILE "echo-op-test" "package"> #<CL-SOURCE-FILE "echo-op-test" "example"> #<CL-SOURCE-FILE "echo-op-test" "main">) #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {10018E1B13}> -------------------------------------------------
Note: 1. "No dependency propagating scheme specified..." warning only occurs once with a fresh Lisp image. It won't occur again with OPERATE :force t or :force :all.
I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on."
This is wrong; I don't know why the manual says that. You still can define a method on COMPONENT-DEPENDS-ON, but there's no reason to unless you have a really weird operation. Writing your own COMPONENT-DEPENDS-ON methods is error-prone and a pain in the ass, so it's much easier to subclass one of the direction-DEPENDS-ON methods and get a built-in method that does the right thing.
But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes?
Honestly, I have no clue why there's an (ECHO-OP . NIL) pair at the head of the COMPONENT-DEPENDS-ON list for an ECHO-OP. But MAP-DIRECT-DEPENDENCIES ignores it. Shrug.
"A method for [INPUT-FILES] is often not needed, since ASDF has a pretty clever default input-files mechanism."
Again, I don't know why the manual says that. You don't need to define INPUT-FILES methods for built-in operations, but you will need to define them for custom operations that take files as input. Most user-defined operations subclass either COMPILE-OP or LOAD-OP, which already have their INPUT-FILES (and OUTPUT-FILES) methods, but I've had to define INPUT-FILES methods in the past.
I see, I'll heed your advice: "For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method."
The (ECHO-OP . NIL) pair at the head of the COMPONENT-DEPENDS-ON list might be the default behaviour. It's present even when I subclass only ASDF:OPERATION (see the REPL output pasted above). With quick grep:
----------- file: action.lisp ------------------- (defmethod component-depends-on ((o operation) (c component)) `(;; Normal behavior, to allow user-specified in-order-to dependencies ,@(cdr (assoc (type-of o) (component-in-order-to c))) ;; For backward-compatibility with ASDF2, any operation that doesn't specify propagation ;; or non-propagation through an appropriate mixin will be downward and sideway. ,@(unless (typep o '(or downward-operation upward-operation sideway-operation selfward-operation non-propagating-operation)) (backward-compatible-depends-on o c)))) ------------------------------------------------- URL: https://gitlab.common-lisp.net/asdf/asdf/-/blob/master/action.lisp#L316
So, all operations are by default downward and sideway unless they are subclass of (OR DOWNWARD-OPERATION UPWARD-OPERATION SIDEWAY-OPERATION SELFWARD-OPERATION NON-PROPAGATING-OPERATION). Since it's sideway, it depends on itself, and thus the (ECHO-OP . NIL) pair?
P.S. For readers, Robert will fix the manual issues. Read his following email: https://mailman.common-lisp.net/pipermail/asdf-devel/2022-April/006689.html and his fix at: https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/207.
Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does?
COMPONENT-PATHNAME.
Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP.
Taking the COMPONENT-PATHNAME as the sole INPUT-FILE can't be the default, because lots of operations do something different. If you think of PREPARE-OP, COMPILE-OP and LOAD-OP as being the "normal" operations on a LISP-SOURCE-FILE, only COMPILE-OP takes the source file as an INPUT-FILE. LOAD-OP takes as an INPUT-FILE the compiled fasl, which is the OUTPUT-FILES of COMPILE-OP. I don't think PREPARE-OP takes any INPUT-FILES at all, but I could be wrong.
I see, I'll need to play around to understand better how INPUT-FILES, PERFORM, and OUTPUT-FILES work together.
One question: How do you know whether to define PERFORM primary method or an :AROUND method for the custom operation class? It's not obvious to me which one to choose for different use cases.
Write an :AROUND method if you're subclassing an existing operation like LOAD-OP to modify its behavior, and you intend to use CALL-NEXT-METHOD to invoke the default behavior for your superclass at some point in your method. This is useful if you want to invoke the next method in a dynamic context e.g. with handlers or restarts bound, or with local bindings of special variables; or if you want to conditionally invoke the next method sometimes but not always. Actually, you can do all of these things in a primary method via CALL-NEXT-METHOD, so now that I think of it, just define a primary method. Write an :AROUND method only if you're defining a mix-in class that other people will subclass which wants to do those things.
If you're subclassing an existing operation and you want to add additional behavior, but you still want the default behavior to run unconditionally in a default dynamic context, define a :BEFORE or :AFTER method as appropriate.
Write a primary method if you're defining a new method, or if you want to completely override the behavior of an existing operation. Or if you want to do the things I said earlier with CALL-NEXT-METHOD.
When in doubt, write a primary method.
I see, thanks for your advice! I'll use my common sense as if working with usual CLOS.
I have a few questions regarding SELFWARD-OPERATION:
1. After reading your previous explanation, I realise that it is used to form a chain of operations. Is it a linear chain or a "network chain" of operations?
A "linear chain" would be something like:
op1 -> op2 -> op3 -> ... -> opN
where one operation has to finish operating before the next operation can begin. So, operations have to done in sequence/series. No parallelisation.
A "network chain" would be something like:
-> op2 -> op3 -> op4 ---- / \ op1 ---> op5 --> op8 ----> op9 \ / -> op6 -> op7 -
where parallelisation is possible. So, op1 has to be done first. Then op5 and (op6 and op7) can be done in parallel, before op8 can be done. Same goes for op9, it has to wait for (op2, op3, op4) + op8.
2. Now, with ECHO-OP done, I want to define a new operation PRINT-OP that prints out the content of the file after ECHO-OP is done. I can define PRINT-OP as subclass of ASDF:SELFWARD-OPERATION. But how can I express the operation dependencies such that ASDF knows PRINT-OP depends on ECHO-OP?
I've look into ASDF:LOAD-OP but I can't see where such a dependency is expressed explicitly? Is the operation dependency defined somewhere implicitly?
-- Regards, zacque
-
Robert Goldman -
zacque -
zacque