Are these reasonable across different Lisp implementations?
E.g., what would it mean to try to invoke a DLL-OP on ACL, Lispworks, or CCL?
It seems like we are advertising a lot of functionality that we do not, in fact, support. I'd like to prune this down in some way, eliminating operations on implementations that don't support them.
thanks, r
On Tue, Sep 8, 2015 at 11:53 AM, Robert Goldman rpgoldman@sift.net wrote:
Are these reasonable across different Lisp implementations?
E.g., what would it mean to try to invoke a DLL-OP on ACL, Lispworks, or CCL?
It seems like we are advertising a lot of functionality that we do not, in fact, support. I'd like to prune this down in some way, eliminating operations on implementations that don't support them.
This is a trick question.
On implementations other than ECL (and maybe also MKCL and CLASP), DLL-OP and LIB-OP will capture the outputs of extensions like CFFI, and not of the main Lisp implementation.
Problem is, the extensions need to cooperate, and CFFI doesn't at this time. i.e. they need to compile to .o before they transform the .o into a .so or .a — then LIB-OP and DLL-OP can collect the .o and build a .a or .so respectively.
PS: if someone fixes CFFI, they should *also* use tmpize-pathname then rename-file-overwriting-target to fix the same race condition that otherwise plagues concurrent invocations of ASDF.
Is any CFFI maintainer listening?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org In its weak form, Utilitarianism sums up as a requirement of observational consistency and behavioral relevance for ethical rules.
On 9/8/15 Sep 8 -12:36 PM, Faré wrote:
On Tue, Sep 8, 2015 at 11:53 AM, Robert Goldman rpgoldman@sift.net wrote:
Are these reasonable across different Lisp implementations?
E.g., what would it mean to try to invoke a DLL-OP on ACL, Lispworks, or CCL?
It seems like we are advertising a lot of functionality that we do not, in fact, support. I'd like to prune this down in some way, eliminating operations on implementations that don't support them.
This is a trick question.
On implementations other than ECL (and maybe also MKCL and CLASP), DLL-OP and LIB-OP will capture the outputs of extensions like CFFI, and not of the main Lisp implementation.
Problem is, the extensions need to cooperate, and CFFI doesn't at this time. i.e. they need to compile to .o before they transform the .o into a .so or .a — then LIB-OP and DLL-OP can collect the .o and build a .a or .so respectively.
PS: if someone fixes CFFI, they should *also* use tmpize-pathname then rename-file-overwriting-target to fix the same race condition that otherwise plagues concurrent invocations of ASDF.
Is any CFFI maintainer listening?
I'm not sure I understand. It sounds like on implementations other than C-based ones, these operations will do something fundamentally different. Or am I misunderstanding? Here's what I think you are saying:
On ECL (and possibly others) after you do a DLL-OP, you get a system that can be loaded as a DLL.
On a "conventional lisp" after you do a DLL-OP, you get a DLL for some subset of the functionality.
So then on ECL you would have a system that you could load by loading the DLL. But on a conventional lisp, if you loaded the DLL, you'd get only a subset of the functionality.
Is that right? If so, it seems that it would be The Wrong Thing to use the same name for both things, because they don't do the same thing.
Cheers, r
: Faré On implementations other than ECL (and maybe also MKCL and CLASP), DLL-OP and LIB-OP will capture the outputs of extensions like CFFI, and not of the main Lisp implementation.
Problem is, the extensions need to cooperate, and CFFI doesn't at this time. i.e. they need to compile to .o before they transform the .o into a .so or .a — then LIB-OP and DLL-OP can collect the .o and build a .a or .so respectively.
PS: if someone fixes CFFI, they should *also* use tmpize-pathname then rename-file-overwriting-target to fix the same race condition that otherwise plagues concurrent invocations of ASDF.
Is any CFFI maintainer listening?
: Robert I'm not sure I understand. It sounds like on implementations other than C-based ones, these operations will do something fundamentally different. Or am I misunderstanding? Here's what I think you are saying:
On ECL (and possibly others) after you do a DLL-OP, you get a system that can be loaded as a DLL.
On a "conventional lisp" after you do a DLL-OP, you get a DLL for some subset of the functionality.
So then on ECL you would have a system that you could load by loading the DLL. But on a conventional lisp, if you loaded the DLL, you'd get only a subset of the functionality.
Is that right? If so, it seems that it would be The Wrong Thing to use the same name for both things, because they don't do the same thing.
Well, the idea is as follows: * LIB-OP collects all the .o files for your system in a .a * DLL-OP collects all the .o files for your system in a .so
Now, on a C-based implementation, "all the .o files" means essentially all your system, whereas on other implementations, it means only C extensions that play the game (currently: none).
The idea is that you could always deliver a system (or a system + dependencies) in either one file (.fasl) or two files (.fasl and .so), depending on whether you're on ECL or another implementation.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The Constitution may not be perfect, but it's a lot better than what we've got!
On 9/8/15 Sep 8 -2:03 PM, Faré wrote:
: Faré On implementations other than ECL (and maybe also MKCL and CLASP), DLL-OP and LIB-OP will capture the outputs of extensions like CFFI, and not of the main Lisp implementation.
Problem is, the extensions need to cooperate, and CFFI doesn't at this time. i.e. they need to compile to .o before they transform the .o into a .so or .a — then LIB-OP and DLL-OP can collect the .o and build a .a or .so respectively.
PS: if someone fixes CFFI, they should *also* use tmpize-pathname then rename-file-overwriting-target to fix the same race condition that otherwise plagues concurrent invocations of ASDF.
Is any CFFI maintainer listening?
: Robert I'm not sure I understand. It sounds like on implementations other than C-based ones, these operations will do something fundamentally different. Or am I misunderstanding? Here's what I think you are saying:
On ECL (and possibly others) after you do a DLL-OP, you get a system that can be loaded as a DLL.
On a "conventional lisp" after you do a DLL-OP, you get a DLL for some subset of the functionality.
So then on ECL you would have a system that you could load by loading the DLL. But on a conventional lisp, if you loaded the DLL, you'd get only a subset of the functionality.
Is that right? If so, it seems that it would be The Wrong Thing to use the same name for both things, because they don't do the same thing.
Well, the idea is as follows:
- LIB-OP collects all the .o files for your system in a .a
- DLL-OP collects all the .o files for your system in a .so
Now, on a C-based implementation, "all the .o files" means essentially all your system, whereas on other implementations, it means only C extensions that play the game (currently: none).
The idea is that you could always deliver a system (or a system + dependencies) in either one file (.fasl) or two files (.fasl and .so), depending on whether you're on ECL or another implementation.
I don't mean to be nitpicky, but this seems deeply wrong to me. It makes the semantics of the operation depend on the user's detailed knowledge of the implementation, and does two substantially different things: on a C-based system it's "return me a library version of the system," and on a conventional implementation, it's "return me a set of ancillary files as a library". Also, IIUC your previous email, the notion of what happens on a non-C-based implementation, basically doesn't work.
So right now, I think this is just an attractive nuisance on non C-based implementations. I'd rather not waste a user's time trying to make them work, or my time trying to maintain them. So I'd prefer we just disable LIB-OP and DLL-OP on implementations where they are not known to work. I.e., the burden of proof is a demonstration that they work, not a "you haven't shown that they DON'T work."
That would mean these would currently be either #+ECL, or possibly #+(or ECL MKCL) (cheap and easy) or would have guards (harder to code) with the same semantics. As other implementations provide support, we could turn them on.
If someone is really passionate about having these on a non-C-based implementation, and can make a strong case for them, which includes (1) redefining the behavior description and (2) demonstrating that they function correctly, I'm open to being convinced. But for now, they should be shut off.
On Tue, Sep 8, 2015 at 3:26 PM, Robert Goldman rpgoldman@sift.net wrote:
Well, the idea is as follows:
- LIB-OP collects all the .o files for your system in a .a
- DLL-OP collects all the .o files for your system in a .so
Now, on a C-based implementation, "all the .o files" means essentially all your system, whereas on other implementations, it means only C extensions that play the game (currently: none).
The idea is that you could always deliver a system (or a system + dependencies) in either one file (.fasl) or two files (.fasl and .so), depending on whether you're on ECL or another implementation.
I don't mean to be nitpicky, but this seems deeply wrong to me. It makes the semantics of the operation depend on the user's detailed knowledge of the implementation, and does two substantially different things: on a C-based system it's "return me a library version of the system," and on a conventional implementation, it's "return me a set of ancillary files as a library". Also, IIUC your previous email, the notion of what happens on a non-C-based implementation, basically doesn't work.
On a C-based system, it gives you a library with BOTH the lisp files *and* the ancillary files linked together. On a non-C-based system, it gives you only the ancillary files.
The semantics is "link all that can be linked".
The fact that on neither implementation does CFFI properly provide ancillary files is secondary. This can and should be fixed.
So right now, I think this is just an attractive nuisance on non C-based implementations. I'd rather not waste a user's time trying to make them work, or my time trying to maintain them. So I'd prefer we just disable LIB-OP and DLL-OP on implementations where they are not known to work. I.e., the burden of proof is a demonstration that they work, not a "you haven't shown that they DON'T work."
Please keep them for 3.1.6, and let's discuss what to do for 3.2 afterwards.
This is a conversation worth having, but I believe it shouldn't delay the release.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org It should be a grammatical if not legal offense to ascribe thoughts, opinions and decisions to "we" without a signed power of attorney.
I spent the evening fixing {,monolithic-}{lib,dll}-op on sbcl and other non-C-linked implementations.
It works, but requires a patched CFFI, that I just posted on cffi-devel. The parts of that patched CFFI that handle linking could be imported into asdf/bundle, but I don't think this would be a good idea for ASDF 3.1.6. Maybe for ASDF 3.2.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org You tell me what you mean by God, and I'll tell you whether I believe in him or not! — E. O. James
Dear Daniel, dear Jean-Claude,
when asdf-ecl was initially written, its load-fasl-op was intended as the default way to load a system. Because of implementation bugs revealed as ASDF improved its testing, this feature was disabled at some point while developing ASDF 3.1. Now that these implementation bugs seem to have been solved, for both ECL and MKCL, the question is: do you guys want me to make load-bundle-op (as it is now named) the default *load-system-operation* on ECL and/or MKCL?
And if so, what test shall I use or not use in determining whether the implementation is recent enough to use this feature without bug? I suppose it's better to check against a recent version of ECL that I can test against this feature than over an older version that may or may not work.
This is probably not a change that should happen in 3.1.6, but probably shortly afterwards.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The kingly office is entitled to no respect. It was originally procured by the highwayman's methods; it remains a perpetuated crime, can never be anything but the symbol of a crime. It is no more entitled to respect than is the flag of a pirate. — Mark Twain
Hello,
I'm in favor of making it the default option. Regarding detecting if it's corrected - it was already OK in ECL 16.0.0, so I suppose it should be the default from lexically detected version 16.0.0 onwards.
Best regards, Daniel
Faré writes:
Dear Daniel, dear Jean-Claude,
when asdf-ecl was initially written, its load-fasl-op was intended as the default way to load a system. Because of implementation bugs revealed as ASDF improved its testing, this feature was disabled at some point while developing ASDF 3.1. Now that these implementation bugs seem to have been solved, for both ECL and MKCL, the question is: do you guys want me to make load-bundle-op (as it is now named) the default *load-system-operation* on ECL and/or MKCL?
And if so, what test shall I use or not use in determining whether the implementation is recent enough to use this feature without bug? I suppose it's better to check against a recent version of ECL that I can test against this feature than over an older version that may or may not work.
This is probably not a change that should happen in 3.1.6, but probably shortly afterwards.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The kingly office is entitled to no respect. It was originally procured by the highwayman's methods; it remains a perpetuated crime, can never be anything but the symbol of a crime. It is no more entitled to respect than is the flag of a pirate. — Mark Twain
Dear François-René,
On Wed, Sep 23, 2015 at 1:21 AM, Faré fahree@gmail.com wrote:
Dear Daniel, dear Jean-Claude,
when asdf-ecl was initially written, its load-fasl-op was intended as the default way to load a system. Because of implementation bugs revealed as ASDF improved its testing, this feature was disabled at some point while developing ASDF 3.1. Now that these implementation bugs seem to have been solved, for both ECL and MKCL, the question is: do you guys want me to make load-bundle-op (as it is now named) the default *load-system-operation* on ECL and/or MKCL?
I have been away from ASDF related concerns long enough for me to be unable to form a precise understanding of what such move would precisely mean right now, sorry. But I will try to push out the door MKCL 1.1.10, the latest maintenance release of the MKCL 1.1.X line, before the end of this month. And as part of that operation I want to upgrade the bundled ASDF to 3.1.5. So I'll have then a great opportunity to get reacquainted with all those load-XXX-op questions, and I should be able to have an informed opinion by then.
And if so, what test shall I use or not use in determining whether the implementation is recent enough to use this feature without bug? I suppose it's better to check against a recent version of ECL that I can test against this feature than over an older version that may or may not work.
Are we talking about something like #'cl:lisp-implementation-version and related facilities? If so then you could be interested in:
#'si:mkcl-major-version #'si:mkcl-minor-version #'si:mkcl-patch-level
that are available since MKCL 1.1.0. They are meant to make this kind of assertion return T:
(typep (read-from-string (si:mkcl-minor-version)) 'integer)
Cheers,
JCB
: Faré
when asdf-ecl was initially written, its load-fasl-op was intended as the default way to load a system. Because of implementation bugs revealed as ASDF improved its testing, this feature was disabled at some point while developing ASDF 3.1. Now that these implementation bugs seem to have been solved, for both ECL and MKCL, the question is: do you guys want me to make load-bundle-op (as it is now named) the default *load-system-operation* on ECL and/or MKCL?
:JCB I have been away from ASDF related concerns long enough for me to be unable to form a precise understanding of what such move would precisely mean right now, sorry. But I will try to push out the door MKCL 1.1.10, the latest maintenance release of the MKCL 1.1.X line, before the end of this month. And as part of that operation I want to upgrade the bundled ASDF to 3.1.5. So I'll have then a great opportunity to get reacquainted with all those load-XXX-op questions, and I should be able to have an informed opinion by then.
The question is whether you prefer to load a previously compiled system via a single .fasb or via plenty of .fas (which wasn't previously working).
Are we talking about something like #'cl:lisp-implementation-version and related facilities? If so then you could be interested in:
#'si:mkcl-major-version #'si:mkcl-minor-version #'si:mkcl-patch-level
Is (lisp-implementation-version) guaranteed to be the concatenation of these, with no trailing data?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org ...so this guy walks into a bar. "The usual, Mr. Descartes?" the barman asked. "I think not," Rene replied, and promptly disappeared.
On Wed, Sep 23, 2015 at 1:10 PM, Faré fahree@gmail.com wrote:
: Faré
when asdf-ecl was initially written, its load-fasl-op was intended as the default way to load a system. Because of implementation bugs revealed as ASDF improved its testing, this feature was disabled at some point while developing ASDF 3.1. Now that these implementation bugs seem to have been solved, for both ECL and MKCL, the question is: do you guys want me to make load-bundle-op (as it is now named) the default *load-system-operation* on ECL and/or MKCL?
:JCB I have been away from ASDF related concerns long enough for me to be unable to form a precise understanding of what such move would precisely mean right now, sorry. But I will try to push out the door MKCL 1.1.10, the latest maintenance release of the MKCL 1.1.X line, before the end of this month. And as part of that operation I want to upgrade the bundled ASDF to 3.1.5. So I'll have then a great opportunity to get reacquainted with all those load-XXX-op questions, and I should be able to have an informed opinion by then.
The question is whether you prefer to load a previously compiled system via a single .fasb or via plenty of .fas (which wasn't previously working).
Put like this I would say that my natural inclination is to say that .fasb should (always?) have precedence over a bunch of .fas if the bunch of .fas provides the same functionality as the .fasb.
Are we talking about something like #'cl:lisp-implementation-version and related facilities? If so then you could be interested in:
#'si:mkcl-major-version #'si:mkcl-minor-version #'si:mkcl-patch-level
Is (lisp-implementation-version) guaranteed to be the concatenation of these, with no trailing data?
Well, the idea is indeed to let #'cl:lisp-implementation-version be free to have some trailing data like perhaps a test level marker ("alpha", "beta", "rc1"...). Otherwise our three little stooges up here would be pretty redundant, wouldn't they?
But currently this returns T:
(string= (lisp-implementation-version) (concatenate 'base-string (si::mkcl-major-version) "." (si::mkcl-minor-version) "." (si::mkcl-patch-level)))
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org ...so this guy walks into a bar. "The usual, Mr. Descartes?" the barman asked. "I think not," Rene replied, and promptly disappeared.
Hi,
Sorry that it's just before the next release. I have some regressions with fix provided to make test-xach-update-bug pass when it comes to working with C code (fix randomizes names which must be predetermined from the C side perspective) so I'm taking that back for now - please don't make bundles the default. I hope to find a viable solution before next ECL release.
Current develop head doesn't pass this test.
Related issues: https://gitlab.com/embeddable-common-lisp/ecl/issues/74 https://gitlab.com/embeddable-common-lisp/ecl/issues/177
Regrads, Daniel
Faré writes:
Dear Daniel, dear Jean-Claude,
when asdf-ecl was initially written, its load-fasl-op was intended as the default way to load a system. Because of implementation bugs revealed as ASDF improved its testing, this feature was disabled at some point while developing ASDF 3.1. Now that these implementation bugs seem to have been solved, for both ECL and MKCL, the question is: do you guys want me to make load-bundle-op (as it is now named) the default *load-system-operation* on ECL and/or MKCL?
And if so, what test shall I use or not use in determining whether the implementation is recent enough to use this feature without bug? I suppose it's better to check against a recent version of ECL that I can test against this feature than over an older version that may or may not work.
This is probably not a change that should happen in 3.1.6, but probably shortly afterwards.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The kingly office is entitled to no respect. It was originally procured by the highwayman's methods; it remains a perpetuated crime, can never be anything but the symbol of a crime. It is no more entitled to respect than is the flag of a pirate. — Mark Twain
-
Daniel Kochmański -
Faré -
Jean-Claude Beaudoin -
Robert Goldman