I *really* like using package-inferred-system for defining systems, but I often find it doesn't behave as I expect when performing operations on it. I'm curious if these behaviors are intended or if the implementation is just incomplete.
I approach package-inferred-system as a way of automatically finding dependencies between files and on third-party systems for a given system I'm writing. Under the hood, I know it makes a new system for every source file, but I consider that just an implementation detail and that when I refer to the primary system, I really mean everything that logically belongs to that system and not just the source files declared in the system's :components option. (Perhaps this assumption just needs to be adjusted on my end).
The difference between expectation and reality often manifests itself in two places for me:
+ When I'm trying to distribute my system with a non-monolithic operation. E.g. `(asdf:perform 'asdf:concatenate-source-op :some-package-inferred-system) results in only the files explicitly declared in the system's :components option being concatenated. (Similar things happen with compile-bundle-op).
+ `(asdf:load-system :some-package-inferred-system :force t)` results in only the files explicitly declared in the system's :components option being reloaded.
I was planning to write my own system class to address these issues, but figured I should see if this is even intended behavior before doing that =).
-Eric
On Thu, Jul 28, 2016 at 12:48 PM, Eric Timmons etimmons@mit.edu wrote:
I *really* like using package-inferred-system for defining systems, but I often find it doesn't behave as I expect when performing operations on it. I'm curious if these behaviors are intended or if the implementation is just incomplete.
I approach package-inferred-system as a way of automatically finding dependencies between files and on third-party systems for a given system I'm writing. Under the hood, I know it makes a new system for every source file, but I consider that just an implementation detail and that when I refer to the primary system, I really mean everything that logically belongs to that system and not just the source files declared in the system's :components option. (Perhaps this assumption just needs to be adjusted on my end).
The difference between expectation and reality often manifests itself in two places for me:
- When I'm trying to distribute my system with a non-monolithic
operation. E.g. `(asdf:perform 'asdf:concatenate-source-op :some-package-inferred-system) results in only the files explicitly declared in the system's :components option being concatenated. (Similar things happen with compile-bundle-op).
- `(asdf:load-system :some-package-inferred-system :force t)` results
in only the files explicitly declared in the system's :components option being reloaded.
I was planning to write my own system class to address these issues, but figured I should see if this is even intended behavior before doing that =).
Indeed, this suggests that plan classes and bundle operations should probably be able to accept a force / force-not argument (or the equivalent for bundle operations) intermediate between t and :all, such as :by-primary, that forces traversal or non-traversal based on the primary system name.
I'm not willing to write it, but I'm willing to review it if and when you do. This probably requires a patch to ASDF itself. Extra props to you if you find a nice extension protocol that allows to move most of that code outside of ASDF itself.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Yesterday I was clever, so I wanted to change the world. Today I am wise, so I am changing myself. — Rumi (1207–1273)
On Thu, Jul 28, 2016 at 11:54 PM, Faré fahree@gmail.com wrote:
On Thu, Jul 28, 2016 at 12:48 PM, Eric Timmons etimmons@mit.edu wrote:
I *really* like using package-inferred-system for defining systems, but I often find it doesn't behave as I expect when performing operations on it. I'm curious if these behaviors are intended or if the implementation is just incomplete.
I approach package-inferred-system as a way of automatically finding dependencies between files and on third-party systems for a given system I'm writing. Under the hood, I know it makes a new system for every source file, but I consider that just an implementation detail and that when I refer to the primary system, I really mean everything that logically belongs to that system and not just the source files declared in the system's :components option. (Perhaps this assumption just needs to be adjusted on my end).
The difference between expectation and reality often manifests itself in two places for me:
- When I'm trying to distribute my system with a non-monolithic
operation. E.g. `(asdf:perform 'asdf:concatenate-source-op :some-package-inferred-system) results in only the files explicitly declared in the system's :components option being concatenated. (Similar things happen with compile-bundle-op).
- `(asdf:load-system :some-package-inferred-system :force t)` results
in only the files explicitly declared in the system's :components option being reloaded.
I was planning to write my own system class to address these issues, but figured I should see if this is even intended behavior before doing that =).
Indeed, this suggests that plan classes and bundle operations should probably be able to accept a force / force-not argument (or the equivalent for bundle operations) intermediate between t and :all, such as :by-primary, that forces traversal or non-traversal based on the primary system name.
I'm not willing to write it, but I'm willing to review it if and when you do. This probably requires a patch to ASDF itself. Extra props to you if you find a nice extension protocol that allows to move most of that code outside of ASDF itself.
Apologies for letting this languish for so long. Real life and work consumed all my cycles for a while.
I'm happy to work on this. Before getting started though, I'd like to get some more feedback on what the default behavior should be. It seems odd that if someone else wants to bundle my system they need to know that it's a package-inferred-system and pass the correct keyword arguments to OPERATE. I think the principle of least surprise would dictate that bundling a package-inferred-system :a would bundle all of its inferred children as well since that's presumably what the developer intended (I can't think of a great reason to bundle just the parent). A corner case here though is what if :a/b is passed to the bundle op by the user. If :a/b isn't a top-level system should the default be to bundle just :a/b, :a/b and :a/b/*, or :a and :a/* ? I don't have a good answer there.
Admittedly, I can't really make the same argument about `:force t` when using LOAD-SYSTEM, because presumably if you're reloading something you're hacking on it and should know that it's a package-inferred-system. However, the part of me that loves symmetry thinks reloading :a and :a/* is the right thing to do and that the new value accepted for :force would stop that behavior.
-Eric
On Thu, Jul 28, 2016 at 12:48 PM, Eric Timmons etimmons@mit.edu wrote:
- When I'm trying to distribute my system with a non-monolithic
operation. E.g. `(asdf:perform 'asdf:concatenate-source-op :some-package-inferred-system) results in only the files explicitly declared in the system's :components option being concatenated. (Similar things happen with compile-bundle-op).
- `(asdf:load-system :some-package-inferred-system :force t)` results
in only the files explicitly declared in the system's :components option being reloaded.
I was planning to write my own system class to address these issues, but figured I should see if this is even intended behavior before doing that =).
Apologies for letting this languish for so long. Real life and work consumed all my cycles for a while.
I'm happy to work on this. Before getting started though, I'd like to get some more feedback on what the default behavior should be. It seems odd that if someone else wants to bundle my system they need to know that it's a package-inferred-system and pass the correct keyword arguments to OPERATE. I think the principle of least surprise would dictate that bundling a package-inferred-system :a would bundle all of its inferred children as well since that's presumably what the developer intended (I can't think of a great reason to bundle just the parent). A corner case here though is what if :a/b is passed to the bundle op by the user. If :a/b isn't a top-level system should the default be to bundle just :a/b, :a/b and :a/b/*, or :a and :a/* ? I don't have a good answer there.
Admittedly, I can't really make the same argument about `:force t` when using LOAD-SYSTEM, because presumably if you're reloading something you're hacking on it and should know that it's a package-inferred-system. However, the part of me that loves symmetry thinks reloading :a and :a/* is the right thing to do and that the new value accepted for :force would stop that behavior.
-Eric
Dear Eric,
first, I'd prefer it if you would write the .script that tests the feature, check that it fails with the current ASDF, and build consensus with the maintainers that the change is the right thing to do.
For force and force-not, it would be a matter of using primary-system-name rather than coerce-name in the correct few places in plan.lisp, notably function action-override-p.
For non-monolithic bundling, that's harder, because that means we can't filter by component type like we do in filtered-sequential-plan, but must instead check the primary-system-name of the systems or the components you depend on.
However, it's not obvious at all what is the Right Thing(tm) regarding bundling package-inferred-systems. What if I write three libraries and two applications all in the same package-inferred-system hierarchy? When I try to bundle them, should I get one bundle per file (which is not very bundly)? Or one bundle for everything that one file depends on under the same hierarchy (then there will be a lot of overlap between those bundles)? Or somehow be able to specify where "this bundle" stops? How?
You should probably be thinking about a coherent interface and specification of what should be done, before you start coding. Coding will be the easy part.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Transported to a surreal landscape, a young girl kills the first woman she meets and then teams up with three complete strangers to kill again." — TV listing for the Wizard of Oz in the Marin Independent Journal
-
Eric Timmons -
Faré