Recent disappearance of asdf:bundle-system.
Hello ASDF devs, I noticed recently that asdf/bundle:bundle-system has disappeared from ASDF. MKCL is/was a user of that function as a convenient entry point to the ASDF bundle facility. My impression was that, along the lines of asdf:load-system and asdf:compile-system, asdf:bundle-system was playing a (small but) useful role as a convenience function. I will most probably simply have to fold that function inside the build code of MKCL if it is to be removed from ASDF for good. Yet, to see a publicly exported function disappear between version 3.1.7.10 and 3.1.7.12 to most likely result in a 3.1.8 release gives pause. Are we really seeing here a public API code breaking modification being done between two patch level releases? BTW, there seems to have been a few other MKCL related breakage issues introduced by the recent wave of changes. I will have to examine each one in turn... Cheers, JCB
On Sep 11, 2016, at 08:31, Jean-Claude Beaudoin <jean.claude.beaudoin@gmail.com> wrote:
Hello ASDF devs,
I noticed recently that asdf/bundle:bundle-system has disappeared from ASDF. MKCL is/was a user of that function as a convenient entry point to the ASDF bundle facility.
[…] For what it is worth, ABCL had an implementation for BUNDLE-SYSTEM that, while suffering from a little bitrot, constructed a “monolithic” FASL from all the compilation units of a give ASDF definition. If BUNDLE-SYSTEM gets resurrected, I would re-work ABCL’s implementation to be more conformant. -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."
Jean-Claude Beaudoin writes:
Hello ASDF devs,
I noticed recently that asdf/bundle:bundle-system has disappeared from ASDF. MKCL is/was a user of that function as a convenient entry point to the ASDF bundle facility.
My impression was that, along the lines of asdf:load-system and asdf:compile-system, asdf:bundle-system was playing a (small but) useful role as a convenience function. I will most probably simply have to fold that function inside the build code of MKCL if it is to be removed from ASDF for good.
Yes, it was my doing, sorry about that. I'll revert that change today and send an appropriate patch. Especially that ABCL wants to use it too. Generally I've seen both functions make-build and bundle-system as deprecated for quite a while in asdf source code. While I didn't want to get rid of make-build, because it's part of the official ECL build API, I've grepped through the libraries in Quicklisp and nobody seemed to use bundle-system.
Yet, to see a publicly exported function disappear between version 3.1.7.10 and 3.1.7.12 to most likely result in a 3.1.8 release gives pause. Are we really seeing here a public API code breaking modification being done between two patch level releases?
It's not patch-level release, but commit level.
BTW, there seems to have been a few other MKCL related breakage issues introduced by the recent wave of changes. I will have to examine each one in turn...
Let me know if you troubleshot this. I think we should revert for mkcl replacement of load-op with load-bundle-op and keep it for ecl and clasp in that case. I'll check on clasp if it doesn't have a regression.
Cheers,
JCB
Regards, Daniel -- Daniel Kochmański ;; aka jackdaniel | Poznań, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
Hey, now all MKCL tests pass as expected. I've also disabled load-bundle-op as a default option (seems like a MKCL bug): https://gitlab.common-lisp.net/asdf/asdf/merge_requests/11 The issue with the system modules is caused by a muss in the systems definitions of MKCL. Namely asd files are bogus – for instance there is (repeated) cmp.asd definition in contrib/ directory pointing and cmp.a file, but the latter isn't present there. `locate-system' takes the cmp.asd from the contrib/ directory and can't inject proper module. I've disabled for mkcl find-system check, but mkcl will fail, if any asd system has "cmp" in `:depends-on' (and this is not a regression, it was like this before and is a problem with mkcl asd files). Long story short – everything is as fine as was before this "wave of change". Best regards, Daniel Daniel Kochmański writes:
Jean-Claude Beaudoin writes:
Hello ASDF devs,
I noticed recently that asdf/bundle:bundle-system has disappeared from ASDF. MKCL is/was a user of that function as a convenient entry point to the ASDF bundle facility.
My impression was that, along the lines of asdf:load-system and asdf:compile-system, asdf:bundle-system was playing a (small but) useful role as a convenience function. I will most probably simply have to fold that function inside the build code of MKCL if it is to be removed from ASDF for good.
Yes, it was my doing, sorry about that. I'll revert that change today and send an appropriate patch. Especially that ABCL wants to use it too.
Generally I've seen both functions make-build and bundle-system as deprecated for quite a while in asdf source code. While I didn't want to get rid of make-build, because it's part of the official ECL build API, I've grepped through the libraries in Quicklisp and nobody seemed to use bundle-system.
Yet, to see a publicly exported function disappear between version 3.1.7.10 and 3.1.7.12 to most likely result in a 3.1.8 release gives pause. Are we really seeing here a public API code breaking modification being done between two patch level releases?
It's not patch-level release, but commit level.
BTW, there seems to have been a few other MKCL related breakage issues introduced by the recent wave of changes. I will have to examine each one in turn...
Let me know if you troubleshot this. I think we should revert for mkcl replacement of load-op with load-bundle-op and keep it for ecl and clasp in that case. I'll check on clasp if it doesn't have a regression.
Cheers,
JCB
Regards, Daniel
-- Daniel Kochmański ;; aka jackdaniel | Poznań, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
On Sun, Sep 11, 2016 at 8:12 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Hey,
now all MKCL tests pass as expected. I've also disabled load-bundle-op as a default option (seems like a MKCL bug):
I'll have to look deeper into that load-bundle-op question...
The issue with the system modules is caused by a muss in the systems definitions of MKCL. Namely asd files are bogus – for instance there is (repeated) cmp.asd definition in contrib/ directory pointing and cmp.a file, but the latter isn't present there. `locate-system' takes the cmp.asd from the contrib/ directory and can't inject proper module. I've disabled for mkcl find-system check, but mkcl will fail, if any asd system has "cmp" in `:depends-on' (and this is not a regression, it was like this before and is a problem with mkcl asd files).
Yep, that's a bug. I had not noticed that bogus cmp.asd before now, that's bad.
Long story short – everything is as fine as was before this "wave of change".
Thanks, I'll give the test suite one more spin as soon as I get an opportunity, just to confirm.
1- While the trivial convenience function bundle-system was removed, the underlying functionality still exists. The function was ill-named legacy of dubious value. Do ABCL users actually use this function as such? 2- On the other hand, its removal is a bit brutal. It might have been better to use the gradual removal strategy we were thinking of in branch obsolete-function-warnings. Maybe there's a case for reversing this removal before next release, and instead completing and merging that branch and using it. I however do not volunteer for the job at this point. 3- I merged part of this pull request into 3.1.7.13. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org What is mind? No matter! What is matter? Never mind! — Bertrand Russell's Grand Mother, In Karl Popper, The Unended Quest On Sun, Sep 11, 2016 at 8:32 AM, Jean-Claude Beaudoin <jean.claude.beaudoin@gmail.com> wrote:
On Sun, Sep 11, 2016 at 8:12 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Hey,
now all MKCL tests pass as expected. I've also disabled load-bundle-op as a default option (seems like a MKCL bug):
I'll have to look deeper into that load-bundle-op question...
The issue with the system modules is caused by a muss in the systems definitions of MKCL. Namely asd files are bogus – for instance there is (repeated) cmp.asd definition in contrib/ directory pointing and cmp.a file, but the latter isn't present there. `locate-system' takes the cmp.asd from the contrib/ directory and can't inject proper module. I've disabled for mkcl find-system check, but mkcl will fail, if any asd system has "cmp" in `:depends-on' (and this is not a regression, it was like this before and is a problem with mkcl asd files).
Yep, that's a bug. I had not noticed that bogus cmp.asd before now, that's bad.
Long story short – everything is as fine as was before this "wave of change".
Thanks, I'll give the test suite one more spin as soon as I get an opportunity, just to confirm.
Faré writes:
1- While the trivial convenience function bundle-system was removed, the underlying functionality still exists. The function was ill-named legacy of dubious value. Do ABCL users actually use this function as such?
bundle-system is a function with a good name, reflecting it's premise – it creates bundled system. I.e it's not less valid than require-system or anything else.
2- On the other hand, its removal is a bit brutal. It might have been better to use the gradual removal strategy we were thinking of in branch obsolete-function-warnings. Maybe there's a case for reversing this removal before next release, and instead completing and merging that branch and using it. I however do not volunteer for the job at this point.
This removal was my mistake, not a brutal "right thing to do".
3- I merged part of this pull request into 3.1.7.13.
Please merge my merge request in full, unless you find a technical problems with it.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org What is mind? No matter! What is matter? Never mind! — Bertrand Russell's Grand Mother, In Karl Popper, The Unended Quest
Regards, Daniel
On Sun, Sep 11, 2016 at 8:32 AM, Jean-Claude Beaudoin <jean.claude.beaudoin@gmail.com> wrote:
On Sun, Sep 11, 2016 at 8:12 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Hey,
now all MKCL tests pass as expected. I've also disabled load-bundle-op as a default option (seems like a MKCL bug):
I'll have to look deeper into that load-bundle-op question...
The issue with the system modules is caused by a muss in the systems definitions of MKCL. Namely asd files are bogus – for instance there is (repeated) cmp.asd definition in contrib/ directory pointing and cmp.a file, but the latter isn't present there. `locate-system' takes the cmp.asd from the contrib/ directory and can't inject proper module. I've disabled for mkcl find-system check, but mkcl will fail, if any asd system has "cmp" in `:depends-on' (and this is not a regression, it was like this before and is a problem with mkcl asd files).
Yep, that's a bug. I had not noticed that bogus cmp.asd before now, that's bad.
Long story short – everything is as fine as was before this "wave of change".
Thanks, I'll give the test suite one more spin as soon as I get an opportunity, just to confirm.
-- Daniel Kochmański ;; aka jackdaniel | Poznań, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
On 2016/9/11 17:09, Faré wrote:
1- While the trivial convenience function bundle-system was removed, the underlying functionality still exists. The function was ill-named legacy of dubious value. Do ABCL users actually use this function as such?
[Sorry for the late reply] As far as I know, no one actively uses BUNDLE-SYSTEM for ABCL, instead using the [ASDF-JAR ABCL contrib][1], which allows a system to (optionally recursively) package all dependencies of an ASDF system into a single binary artifact (a jar file containing all the code). If I understand things conceptually correctly, if their were a stable BUNDLE-SYSTEM, we would move the code for ASDF-JAR to use those interfaces. [1]: http://abcl.org/trac/browser/trunk/abcl/contrib/asdf-jar -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On 10/16/16 Oct 16 -1:31 AM, Mark Evenson wrote:
On 2016/9/11 17:09, Faré wrote:
1- While the trivial convenience function bundle-system was removed, the underlying functionality still exists. The function was ill-named legacy of dubious value. Do ABCL users actually use this function as such?
[Sorry for the late reply]
As far as I know, no one actively uses BUNDLE-SYSTEM for ABCL, instead using the [ASDF-JAR ABCL contrib][1], which allows a system to (optionally recursively) package all dependencies of an ASDF system into a single binary artifact (a jar file containing all the code).
If I understand things conceptually correctly, if their were a stable BUNDLE-SYSTEM, we would move the code for ASDF-JAR to use those interfaces.
[1]: http://abcl.org/trac/browser/trunk/abcl/contrib/asdf-jar
So BUNDLE-SYSTEM has been restored, but the comment about its deprecation suggests that we should revisit its arglist to make it useful, if we are not going to remove it. TBH, I don't understand what bundle-system is supposed to do, versus, for example LOAD-SYSTEM. There seem to be so many different ways to bundle and kinds of bundle that this function seems too ambiguous. The function is not documented, and reading the code doesn't help me (as someone who doesn't routinely use bundle ops): it defers to DELIVER-ASD-OP, but that operation is documented as: "produce an asd file for delivering the system as a single fasl" which suggests that it makes a .asd file, and doesn't bundle anything at all. So.... help?
Robert Goldman writes:
On 10/16/16 Oct 16 -1:31 AM, Mark Evenson wrote:
On 2016/9/11 17:09, Faré wrote:
1- While the trivial convenience function bundle-system was removed, the underlying functionality still exists. The function was ill-named legacy of dubious value. Do ABCL users actually use this function as such?
[Sorry for the late reply]
As far as I know, no one actively uses BUNDLE-SYSTEM for ABCL, instead using the [ASDF-JAR ABCL contrib][1], which allows a system to (optionally recursively) package all dependencies of an ASDF system into a single binary artifact (a jar file containing all the code).
If I understand things conceptually correctly, if their were a stable BUNDLE-SYSTEM, we would move the code for ASDF-JAR to use those interfaces.
[1]: http://abcl.org/trac/browser/trunk/abcl/contrib/asdf-jar
So BUNDLE-SYSTEM has been restored, but the comment about its deprecation suggests that we should revisit its arglist to make it useful, if we are not going to remove it.
TBH, I don't understand what bundle-system is supposed to do, versus, for example LOAD-SYSTEM. There seem to be so many different ways to bundle and kinds of bundle that this function seems too ambiguous.
The function is not documented, and reading the code doesn't help me (as someone who doesn't routinely use bundle ops): it defers to DELIVER-ASD-OP, but that operation is documented as:
"produce an asd file for delivering the system as a single fasl"
which suggests that it makes a .asd file, and doesn't bundle anything at all.
So.... help?
Afaik bundle-system creates one fasl for a whole system (if monolithic - for a system and its dependencies). It may be loaded afterwards with simple load. What it lacks is a `:move-here' key parameter or a similar mechanism. I wouldn't advise to use it, because of an arbitrary decision by Fare to break the API contract (by pushing for removal of `make-build' and `bundle-system' and promoting `program-op' and `deliver-asd-op') - you'll have your api broken with the next or one after next ASDF release. Best regards, Daniel -- Daniel Kochmański ;; aka jackdaniel | Przemyśl, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
So BUNDLE-SYSTEM has been restored, but the comment about its deprecation suggests that we should revisit its arglist to make it useful, if we are not going to remove it.
My "no-operation-initargs" branch, that you (Robert) agreed to merge after the next release, does away with bundle-system.
The function is not documented, and reading the code doesn't help me (as someone who doesn't routinely use bundle ops): it defers to DELIVER-ASD-OP, but that operation is documented as:
"produce an asd file for delivering the system as a single fasl"
which suggests that it makes a .asd file, and doesn't bundle anything at all.
It uses compile-bundle-op (née fasl-op) to create a single .fasl for the entire system, then also creates a .asd file that will load that .fasl file. But there is no good place to store that .asd file least it overwrites the source .asd file, and no good place to store the .fasl file besides the per-implementation fasl cache. The whole interface is lacking, and is not directly usable as is, though it isn't too hard for users to write a wrapper that copy or redirect the output where they want it to be, using deliver-asd-op directly and querying asdf:output-files. Thus, the function has no purpose and should be removed.
Afaik bundle-system creates one fasl for a whole system (if monolithic - for a system and its dependencies). It may be loaded afterwards with simple load. What it lacks is a `:move-here' key parameter or a similar mechanism.
The bundle-system function does not accept a monolithic keyword.
I wouldn't advise to use it, because of an arbitrary decision by Fare to break the API contract (by pushing for removal of `make-build' and `bundle-system' and promoting `program-op' and `deliver-asd-op') - you'll have your api broken with the next or one after next ASDF release.
My decision is not arbitrary. Make-build violates ASDF invariants. I was a clever hack on top of ASDF1, but not a viable design. I will die soon. I've warned Juan Jo, I've warned you. ECL hackers have had years to move off it. An operation class MUST NOT have any instance slots. Please put component-specific flags in component slots, and global flags in global variables. Even if far future extensions ever have instance slots in operation, they shall not be for component-specific flags. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Stop cultural appropriation! Using arabic numbers? Unless an Arab, 3 years in jail. Using roman letters? Unless a Roman, III years in jail.
On 11:43 Sun 16 Oct , Faré wrote:
So BUNDLE-SYSTEM has been restored, but the comment about its deprecation suggests that we should revisit its arglist to make it useful, if we are not going to remove it.
My "no-operation-initargs" branch, that you (Robert) agreed to merge after the next release, does away with bundle-system.
The function is not documented, and reading the code doesn't help me (as someone who doesn't routinely use bundle ops): it defers to DELIVER-ASD-OP, but that operation is documented as:
"produce an asd file for delivering the system as a single fasl"
which suggests that it makes a .asd file, and doesn't bundle anything at all.
It uses compile-bundle-op (née fasl-op) to create a single .fasl for the entire system, then also creates a .asd file that will load that .fasl file. But there is no good place to store that .asd file least it overwrites the source .asd file, and no good place to store the .fasl file besides the per-implementation fasl cache. The whole interface is lacking, and is not directly usable as is, though it isn't too hard for users to write a wrapper that copy or redirect the output where they want it to be, using deliver-asd-op directly and querying asdf:output-files. Thus, the function has no purpose and should be removed.
I currently use compile-bundle-op and deliver-asd-op (in fact expect a bugfix soon) for creating read-only loadable systems for Nix (I previously tried just managing the output translations but found it to get too hairy; with the bundles no changes to the output translations are necessary). Are these two operations here to stay? -Jason
On 9/11/16 Sep 11 -2:44 AM, Daniel Kochmański wrote:
Generally I've seen both functions make-build and bundle-system as deprecated for quite a while in asdf source code. While I didn't want to get rid of make-build, because it's part of the official ECL build API, I've grepped through the libraries in Quicklisp and nobody seemed to use bundle-system.
I think this is an example of Quicklisp being an imperfect measure for the level of use of the API. This happens every now and then and I want to take the opportunity to reiterate that "The community of CL users" != "the community of QL users." For example: Quicklisp isn't really about bundling *applications*, it's about assembling systems out of components, not about distribution. Of course, code *might* slip in to QL that is delivery-related, but that would be just coincidental. Also: Code used in non-open source projects, or projects that for one reason or another haven't entered into QL cannot be broken willy-nilly. No API function should be removed, or modified in its nature without clearing it through ASDF-devel. Thanks!
participants (6)
-
Daniel Kochmański -
Faré -
Jason Miller -
Jean-Claude Beaudoin -
Mark Evenson -
Robert Goldman