Right now there are hooks in ASDF which are used by ECL in a perhaps too complicated way. More precisely, ECL requires to make an object file and a FASL file out of every compiled file, so that other operations (LOAD-FASL-OP, monolithic building, standalone executables, etc) become easier. Right now this is done using an :AFTER method on PERFORM. I suggest this patch (already in ECL's tree) which removes that method, removes the upgradability hacks and insteads wraps around COMPILE-FILE* (which now becomes redefinable at run time).
Juanjo
Right after sending the patch I realized two things:
- A debug statement was left out (TRACE ...) - It might make sense to save the old value of COMPILE-FILE*
Attached improved diff against recent ASDF tree.
Juanjo
On 4 February 2011 18:11, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
Attached improved diff against recent ASDF tree.
OK, I'm going to apply this one. Are there any required changes in asdf-ecl.lisp?
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] "I carry a gun because a cop is too heavy." — R. Lee Wrights
On Sat, Feb 5, 2011 at 12:30 AM, Faré fahree@gmail.com wrote:
On 4 February 2011 18:11, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
Attached improved diff against recent ASDF tree.
OK, I'm going to apply this one. Are there any required changes in asdf-ecl.lisp?
I believe not. But I am still working on this -- the goal is to be able to plug in a second and different compiler (bytecodes) and let ASDF work with both.
Juanjo
On 5 February 2011 02:55, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
OK, I'm going to apply this one. Are there any required changes in asdf-ecl.lisp?
I believe not. But I am still working on this -- the goal is to be able to plug in a second and different compiler (bytecodes) and let ASDF work with both.
I massaged your patch (notably made compile-file* a gf to move your code to an :around method rather than a manual wrapper), tested it, and committed as 2.012.5. Enjoy!
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Pacifism is a shifty doctrine under which a man accepts the benefits of the social group without being willing to pay — and claims a halo for his dishonesty. — Robert Heinlein
On Sat, Feb 5, 2011 at 4:06 PM, Faré fahree@gmail.com wrote:
I massaged your patch (notably made compile-file* a gf to move your code to an :around method rather than a manual wrapper), tested it, and committed as 2.012.5. Enjoy!
Fare, could you please go back to the other method I posted. This one does not allow one to *CHANGE* the way COMPILE-FILE* behaves.
I profoundly dislike generic functions API precisely for this reason. There is a base function, someone adds an around method and what about the next around? How do you remove the previous one?
The other solution based on a simple closure is simpler, has a smaller overhead and allows one to fall back to the original method, reinstating COMPILE-FILE* definition.
Juanjo
On 5 February 2011 11:13, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
On Sat, Feb 5, 2011 at 4:06 PM, Faré fahree@gmail.com wrote: Fare, could you please go back to the other method I posted. This one does not allow one to *CHANGE* the way COMPILE-FILE* behaves.
Do you need to change it dynamically? Why not just have a static #+ecl or something? I don't understand the use case.
I profoundly dislike generic functions API precisely for this reason. There is a base function, someone adds an around method and what about the next around? How do you remove the previous one?
Since all this is ECL-specific (so far), I suppose you could use ADVISE, or whatever ECL-specific hooking mechanism exists, or delete-method calls.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Every technique is first developed, then used, important, obsolete, normalized, and finally understood.
On Sat, Feb 5, 2011 at 5:18 PM, Faré fahree@gmail.com wrote:
On 5 February 2011 11:13, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
On Sat, Feb 5, 2011 at 4:06 PM, Faré fahree@gmail.com wrote: Fare, could you please go back to the other method I posted. This one
does
not allow one to *CHANGE* the way COMPILE-FILE* behaves.
Do you need to change it dynamically? Why not just have a static #+ecl or something? don't understand the use case.
Yes, it has to be changeable if the user decides to switch compilers, for whatever reason. To name one, Windows users will get by default a bytecodes compiler shipped in, and will have to switch to the lisp->C compiler at run time if they have MSVC around (which is rarely the case). And the converse should be possible: Linux users could have lisp->C preloaded and switch to bytecodes compilation for a session. All this should be doable with the same ASDF image.
As you see, it would even be better to have a global variable ASDF:*COMPILE-FILE-FUNCTION* specify who does what instead of having us mess with function definitions.
Since all this is ECL-specific (so far), I suppose you could use ADVISE, or whatever ECL-specific hooking mechanism exists, or delete-method calls.
But do you realize, that precisely because this is very ECL specific, there is no need to complicate COMPILE-FILE* and the way we have to interfere with it.
Design of ASDF should expose only a minimal, well thought out API. Making COMPILE-FILE* a generic function, when the types of arguments is fixed, just for the sake of :AROUND methods is absurd. It forces non portable code (I call MOP nonportable) just to change/delete a feature.
If you want me to put it another way. The patch I submitted was just to get rid of an :AROUND method, and you just forced me to add another one + MOP code in my own system.
Please, reconsider this,
Juanjo
Do you need to change it dynamically? Why not just have a static #+ecl or something? don't understand the use case.
Yes, it has to be changeable if the user decides to switch compilers, for whatever reason. To name one, Windows users will get by default a bytecodes compiler shipped in, and will have to switch to the lisp->C compiler at run time if they have MSVC around (which is rarely the case). And the converse should be possible: Linux users could have lisp->C preloaded and switch to bytecodes compilation for a session. All this should be doable with the same ASDF image.
What about instead having a conditional based on a special variable (that probably already exists in some ECL guts) to decide whether to compile using one method or the other at runtime? e.g. (if ecl:*use-bytecode-compiler* ... (call-next-method))
As you see, it would even be better to have a global variable ASDF:*COMPILE-FILE-FUNCTION* specify who does what instead of having us mess with function definitions.
I'm still not convinced.
Since all this is ECL-specific (so far), I suppose you could use ADVISE, or whatever ECL-specific hooking mechanism exists, or delete-method calls.
But do you realize, that precisely because this is very ECL specific, there is no need to complicate COMPILE-FILE* and the way we have to interfere with it.
I personally think that setf fdefinition is pretty complicated already.
Design of ASDF should expose only a minimal, well thought out API. Making COMPILE-FILE* a generic function, when the types of arguments is fixed, just for the sake of :AROUND methods is absurd. It forces non portable code (I call MOP nonportable) just to change/delete a feature.
A :around method is a portable way of achieving the result (no MOP required), and since you're method is #+ecl there is no problem with non-portability in changing/deleting the feature. Though once again, a conditional is probably better called for, since it just works without the user having to make sure he calls hooks after modifying a variable.
If you want me to put it another way. The patch I submitted was just to get rid of an :AROUND method, and you just forced me to add another one + MOP code in my own system.
Since you're only showing me half the code, I can't judge. As far as the code I'm seeing and maintaining goes, I made my judgment.
Please, reconsider this,
Please show me your code if you want me to reconsider.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] In a culturally confident age, the British in India were faced with the practice of "suttee" — the tradition of burning widows on the funeral pyres of their husbands. General Sir Charles Napier was impeccably multicultural: "You say that it is your custom to burn widows. Very well. We also have a custom: when men burn a woman alive, we tie a rope around their necks and we hang them. Build your funeral pyre; beside it, my carpenters will build a gallows. You may follow your custom. And then we will follow ours."
On Sat, Feb 5, 2011 at 7:17 PM, Faré fahree@gmail.com wrote:
Please show me your code if you want me to reconsider.
Please understand that all this is about not *hardcoding* in ASDF what the compiler should or should not do. All you are suggesting is about reader conditionalization (that means one ASDF file per compiler), hardcoding behavior based on externally defined flags (which may or may not exist at all in older or later releases), and other things that overcomplicate the problem.
Juanjo
On Sat, Feb 5, 2011 at 7:27 PM, Juan Jose Garcia-Ripoll < juanjose.garciaripoll@googlemail.com> wrote:
On Sat, Feb 5, 2011 at 7:17 PM, Faré fahree@gmail.com wrote:
Please show me your code if you want me to reconsider.
Please understand that all this is about not *hardcoding* in ASDF what the compiler should or should not do. All you are suggesting is about reader conditionalization (that means one ASDF file per compiler), hardcoding behavior based on externally defined flags (which may or may not exist at all in older or later releases), and other things that overcomplicate the problem.
Argghh this does not work at all. I did not realize that ASDF's compilation now uses a temporary file name. That is really unfortunate because it means we can not really wrap inside compile-file* and also not around compile-file*, and we have to go back to the level of PERFORM.
Attached is a diff where this customization is thus introduced at the only place where it makes sense, and it is done using a single special variable that can be dynamically changed. Seems to work with quicklisp
Juanjo
On 5 February 2011 17:14, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
Please understand that all this is about not *hardcoding* in ASDF what the compiler should or should not do. All you are suggesting is about reader conditionalization (that means one ASDF file per compiler), hardcoding behavior based on externally defined flags (which may or may not exist at all in older or later releases), and other things that overcomplicate the problem.
Where is the behavior to be coded? Surely it must be coded somewhere. Is it unreasonable to upgrade ASDF when new code is required, and to change few boolean controls to select which branch of the code applies?
Argghh this does not work at all. I did not realize that ASDF's compilation now uses a temporary file name. That is really unfortunate because it means we can not really wrap inside compile-file* and also not around compile-file*, and we have to go back to the level of PERFORM.
Or perhaps we could factor ASDF to do renaming for all client methods in a more transparent way - problem being it would require invalidating existing extensions.
Attached is a diff where this customization is thus introduced at the only place where it makes sense, and it is done using a single special variable that can be dynamically changed. Seems to work with quicklisp
Here's a counter-proposal.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] To take five and return four, isn't giving — it's stealing. To take five by force, and return four under condition of obedience, is worse than stealing — it's enslaving. When most of these four are "returned" in the form of monopolized "services", charged a hefty price despite their inferior quality, and mixed with a large dose of propaganda — it's Government.
On Sun, Feb 6, 2011 at 10:23 PM, Faré fahree@gmail.com wrote:
Where is the behavior to be coded? Surely it must be coded somewhere. Is it unreasonable to upgrade ASDF when new code is required, and to change few boolean controls to select which branch of the code applies?
Please, understand, I do not want to commit to a particular behavior right now. I do not know yet how I am going to code the bytecodes compiler, or if I will allow multiple variants or whether I will have something which is build-fasl-like in a third phase of development.
I want freedom on developing my ECL side of the code while preserving compatibility and minimal intrusiveness. I do not want to have to come here and ask for every single change every time I make different design choice, or even worse, keeping track of versions. Pluggin in into ASDF should be done from my own extensions (asdf-ecl and new compiler backends) and not by hardcoding behavior into ASDF where it does not belong.
I think what I am suggesting is pretty reasonable.
1* COMPILE-FILE* right now has a very well defined purpose, (compile a single lisp file with error handlers) which I believe should not be touched if it is going to be exported as a tool. Move back from method to function. 2* Right now the only ECL-dependent behavior is in PERFORM when applied on COMPILE-OP and on OUTPUT-FILES acting on the same operator. Get these changes away by default. 3* The ECL-specific part only makes sense and only is needed when combined when asdf-ecl.lisp, which itself only makes sense when used with the current C compiler. This can be moved to asdf-ecl. 4* Allow a simple way to plug into the compile-op actions. An :around method is too complicated because there may be more layers and using MOP is both nonportable, non-future proof, and a mess -- what methods to add, what to remove? bootstrapping problems, versioning...
Everything is atached to this email. Just one single hook (a special variable), removed all ECL-specific logic from asdf.lisp, and back into asdf-ecl.lisp, where it belongs, non-intrusively.
Juanjo
On 6 February 2011 16:58, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
Please, understand, I do not want to commit to a particular behavior right now. I do not know yet how I am going to code the bytecodes compiler, or if I will allow multiple variants or whether I will have something which is build-fasl-like in a third phase of development.
OK. But whatever you do will probably require overriding the code in asdf-ecl.lisp, at which point you need to reload asdf, anyway.
I want freedom on developing my ECL side of the code while preserving compatibility and minimal intrusiveness. I do not want to have to come here and ask for every single change every time I make different design choice, or even worse, keeping track of versions. Pluggin in into ASDF should be done from my own extensions (asdf-ecl and new compiler backends) and not by hardcoding behavior into ASDF where it does not belong.
Makes sense.
I think what I am suggesting is pretty reasonable.
1* COMPILE-FILE* right now has a very well defined purpose, (compile a single lisp file with error handlers) which I believe should not be touched if it is going to be exported as a tool. Move back from method to function.
OK.
2* Right now the only ECL-dependent behavior is in PERFORM when applied on COMPILE-OP and on OUTPUT-FILES acting on the same operator. Get these changes away by default.
OK.
3* The ECL-specific part only makes sense and only is needed when combined when asdf-ecl.lisp, which itself only makes sense when used with the current C compiler. This can be moved to asdf-ecl.
OK.
4* Allow a simple way to plug into the compile-op actions. An :around method is too complicated because there may be more layers and using MOP is both nonportable, non-future proof, and a mess -- what methods to add, what to remove? bootstrapping problems, versioning...
OK, OK.
Everything is atached to this email. Just one single hook (a special variable), removed all ECL-specific logic from asdf.lisp, and back into asdf-ecl.lisp, where it belongs, non-intrusively.
See ASDF 2.012.6.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Be inconsistent — but don't do it all the time. — Unknown
On 2/6/11 Feb 6 -3:23 PM, Faré wrote:
On 5 February 2011 17:14, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
Please understand that all this is about not *hardcoding* in ASDF what the compiler should or should not do. All you are suggesting is about reader conditionalization (that means one ASDF file per compiler), hardcoding behavior based on externally defined flags (which may or may not exist at all in older or later releases), and other things that overcomplicate the problem.
Where is the behavior to be coded? Surely it must be coded somewhere. Is it unreasonable to upgrade ASDF when new code is required, and to change few boolean controls to select which branch of the code applies?
Argghh this does not work at all. I did not realize that ASDF's compilation now uses a temporary file name. That is really unfortunate because it means we can not really wrap inside compile-file* and also not around compile-file*, and we have to go back to the level of PERFORM.
Or perhaps we could factor ASDF to do renaming for all client methods in a more transparent way - problem being it would require invalidating existing extensions.
That might be nice --- we have been having some issues with protobuf and ASDF 2, because of having a two stage process of generation of lisp from protocol buffer and then fasl from lisp. I have a few cases where this happens, because it's easy to generate lisp from an abstract specification language. Hm... This should almost be a FAQ.
cheers, r
-
Faré -
Juan Jose Garcia-Ripoll -
Robert Goldman