An issue that came up discussing ASDF-FLV with Didier has to do with outside modification to exported ASDF generic functions. I think it's unambiguously reasonable to say "if you mess with unexported functions, generic or otherwise, you're on your own, and could break things arbitrarily."
But I don't think ASDF has established clear expectations about extending methods of exported ASDF generic functions. I think there's been an unspoken assumption that if ASDF generic functions are to be extended, particularly by `:around`, `:before`, and `:after` methods, there should be some novel class introduced that will be used to select extensions. So, for example, it's fine to wrap or write new methods for `perform`, but not for existing component and operation classes. Doing so means that a change one might think appropriate for one's own library bleeds out into other code that one has no way of predicting, and also has the potential to collide with ASDF's own code or other people's ASDF extensions.
Those are good reasons for this convention -- no modification without introducing new classes for method dispatch -- but ASDF-FLV is an example of a system that wants to introspect about the entire running lisp image. I'd be interested in hearing what people think about this kind of case. Should we introduce, for example, an `ASDF` object that extenders could add mixins to in order to enable modifications to ASDF behavior that aren't limited to a single library?
Please post any thoughts you have about this idea.
What are the methods defined by asdf-flv?
IIRC, POIU also has a few :around methods (it also doesn't properly support ASDF 3.3 phases).
-#f
On Tue, Apr 23, 2024, 17:24 Robert Goldman rpgoldman@sift.net wrote:
An issue that came up discussing ASDF-FLV with Didier has to do with outside modification to exported ASDF generic functions. I think it's unambiguously reasonable to say "if you mess with unexported functions, generic or otherwise, you're on your own, and could break things arbitrarily."
But I don't think ASDF has established clear expectations about extending methods of exported ASDF generic functions. I think there's been an unspoken assumption that if ASDF generic functions are to be extended, particularly by :around, :before, and :after methods, there should be some novel class introduced that will be used to select extensions. So, for example, it's fine to wrap or write new methods for perform, but not for existing component and operation classes. Doing so means that a change one might think appropriate for one's own library bleeds out into other code that one has no way of predicting, and also has the potential to collide with ASDF's own code or other people's ASDF extensions.
Those are good reasons for this convention -- no modification without introducing new classes for method dispatch -- but ASDF-FLV is an example of a system that wants to introspect about the entire running lisp image. I'd be interested in hearing what people think about this kind of case. Should we introduce, for example, an ASDF object that extenders could add mixins to in order to enable modifications to ASDF behavior that aren't limited to a single library?
Please post any thoughts you have about this idea.
François-René ÐVB Rideau écrivait:
What are the methods defined by asdf-flv?
In order to support file local variables, ASDF-FLV does this:
(defmethod asdf:perform :around ((operation asdf:load-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
(defmethod asdf:perform :around ((operation asdf:compile-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
The problem, of course, is that loading ASDF-FLV will override any existing method with the same signature, and conversely, anything else loaded afterward will override ASDF-FLV.
ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that subclass, but that would defeat its pervasiveness and make it more difficult to have multiple extensions working together. Hence Robert's idea of mixins. My own idea was to have exported operations use a special method combination allowing multiple methods with the same signature.
On 24 Apr 2024, at 3:39, Didier Verna wrote:
François-René ÐVB Rideau écrivait:
What are the methods defined by asdf-flv?
In order to support file local variables, ASDF-FLV does this:
(defmethod asdf:perform :around ((operation asdf:load-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
(defmethod asdf:perform :around ((operation asdf:compile-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
The problem, of course, is that loading ASDF-FLV will override any existing method with the same signature, and conversely, anything else loaded afterward will override ASDF-FLV.
ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that subclass, but that would defeat its pervasiveness and make it more difficult to have multiple extensions working together. Hence Robert's idea of mixins. My own idea was to have exported operations use a special method combination allowing multiple methods with the same signature.
The one thing that worries me about that is that I don't know how well all the supported lisp implementations handle defining new method combinations.
But otherwise I agree that this would be an appropriate thing to do.
Funny, but ASDF 1 at some point (commit aa52ad22 1.128 to 1.636) up was defining its own method combination, so users could write their own methods without overwriting ASDF's (but still overwriting each other's).
In practice it was only used for two :around methods for perform, with one being renamed perform-with-restarts in the end. It may also have helped with portability to gcl, that was added at the same time.
-#f
On Thu, Apr 25, 2024, 19:33 Robert Goldman rpgoldman@sift.net wrote:
On 24 Apr 2024, at 3:39, Didier Verna wrote:
François-René ÐVB Rideau écrivait:
What are the methods defined by asdf-flv?
In order to support file local variables, ASDF-FLV does this:
(defmethod asdf:perform :around ((operation asdf:load-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
(defmethod asdf:perform :around ((operation asdf:compile-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
The problem, of course, is that loading ASDF-FLV will override any existing method with the same signature, and conversely, anything else loaded afterward will override ASDF-FLV.
ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that subclass, but that would defeat its pervasiveness and make it more difficult to have multiple extensions working together. Hence Robert's idea of mixins. My own idea was to have exported operations use a special method combination allowing multiple methods with the same signature.
The one thing that worries me about that is that I don't know how well all the supported lisp implementations handle defining new method combinations.
But otherwise I agree that this would be an appropriate thing to do.
On Thu, Apr 25, 2024 at 9:11 PM Faré fahree@gmail.com wrote:
Funny, but ASDF 1 at some point (commit aa52ad22 1.128 to 1.636) up was defining its own method combination, so users could write their own methods without overwriting ASDF's (but still overwriting each other's).
In practice it was only used for two :around methods for perform, with one being renamed perform-with-restarts in the end. It may also have helped with portability to gcl, that was added at the same time.
Uh, I meant *removing* that method combination may have helped with portability to gcl.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “Why is there only one Monopolies Commission?” — Lord Sutch
François-René ÐVB Rideau écrivait:
What are the methods defined by asdf-flv?
In order to support file local variables, ASDF-FLV does this:
(defmethod asdf:perform :around ((operation asdf:load-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
(defmethod asdf:perform :around ((operation asdf:compile-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
The problem, of course, is that loading ASDF-FLV will override any existing method with the same signature, and conversely, anything else loaded afterward will override ASDF-FLV.
ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that subclass, but that would defeat its pervasiveness and make it more difficult to have multiple extensions working together. Hence Robert's idea of mixins. My own idea was to have exported operations use a special method combination allowing multiple methods with the same signature.
1. I once implemented a more general mechanism for binding syntax variables around compilation, which must be part of the syntax-control branch (that probably needs a lot of love rebasing it on top of the latest ASDF). Syntax variables include *readtable*, *read-base*, *read-default-float-format*, etc., as well as any user-defined variables to similar purposes in user extensions.
2. However, note this only has guaranteed semantics if the effects of those variables are at read-time / macroexpansion-time / compile-time. Features such as linking (on ECL, MKCL) or fasl concatenation (on about every other platform) are incompatible with relying on effects from binding variables around individual lisp or fasl files. Of course, you could declare "my system and any system that depends on it is incompatible with linking and fasl concatenation".
3. As for having bindings around compiling individual files, there is already the around-compile hook, that you could use or override or define methods for, to bind variables around such compilation. You could use this interface, and make it more extensible if you want.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “Those who cannot remember the past are condemned to repeat it.” — George Santayana
On Wed, Apr 24, 2024 at 4:51 AM Didier Verna didier@didierverna.net wrote:
François-René ÐVB Rideau écrivait:
What are the methods defined by asdf-flv?
In order to support file local variables, ASDF-FLV does this:
(defmethod asdf:perform :around ((operation asdf:load-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
(defmethod asdf:perform :around ((operation asdf:compile-op) (file asdf:cl-source-file)) "Establish new dynamic bindings for file-local variables." (progv *file-local-variables* (mapcar #'symbol-value *file-local-variables*) (call-next-method)))
The problem, of course, is that loading ASDF-FLV will override any existing method with the same signature, and conversely, anything else loaded afterward will override ASDF-FLV.
ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that subclass, but that would defeat its pervasiveness and make it more difficult to have multiple extensions working together. Hence Robert's idea of mixins. My own idea was to have exported operations use a special method combination allowing multiple methods with the same signature.
-- Resistance is futile. You will be jazzimilated.
Jazz site: http://www.didierverna.com Other sites: http://www.didierverna.info
-
Didier Verna -
Didier Verna -
Faré -
Robert Goldman