I have been working on enforcing assumptions recently added to the ASDF docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD.
I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on OPERATIONs in the ASDF codebase itself.
Cheers, r
1- The requirement to always use make-operation currently only applies to code within ASDF itself and well-behaved extensions. Before you enforce it more widely, you have to make sure no one in Quicklisp does it. You could have shared initialize check that: a) no instance of that class with given initargs exists yet in *operations* and hopefully b) there initargs are always null (goodbye, make-build!)
2- No, there was never a requirement that defsystem should only be used within a .asd. Actually, the test system relies heavily on the opposite. The requirement is that .asd files be loaded in the correct context, by load-asd -- notably, the correct *package* must be bound, the correct readtable, etc.
On Fri, Sep 23, 2016, 09:49 Robert Goldman rpgoldman@sift.net wrote:
I have been working on enforcing assumptions recently added to the ASDF docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD.
I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on OPERATIONs in the ASDF codebase itself.
Cheers, r
On 9/23/16 Sep 23 -9:52 AM, Faré wrote:
1- The requirement to always use make-operation currently only applies to code within ASDF itself and well-behaved extensions. Before you enforce it more widely, you have to make sure no one in Quicklisp does it. You could have shared initialize check that: a) no instance of that class with given initargs exists yet in *operations* and hopefully b) there initargs are always null (goodbye, make-build!)
Your docs clearly say that these MUST be built this way. If you want to say "must" then I will enforce "must." If you want "may," we can have "may" as in "For more efficient functioning, please use MAKE-OPERATION to create all operation instances."
Please articulate why you say "All operation instances MUST be created through this function"? I like the idea of having a clean API, but I also note that there is code that effectively interns the results of MAKE-OPERATION. What is the design rationale for this? Do you wish to be able to EQ-compare them? If not, why do you intern these objects in the *OPERATIONS* table? Is it really so expensive to create an OPERATION that we need to memoize? Or is it because we simply create so many, and this improves GC behavior?
WRT Quicklisp: I do NOT accept that I have to check all of Quicklisp for ANYTHING. I am willing to be INFORMED by results of such checks, but I do NOT have time to make such checks.
I don't expect other people to test my SIFT code, and I don't have time to check arbitrary CL libraries, much less arbitrary CL libraries on arbitrary implementations and operating systems.
So, no. If someone else wants to test other things.
2- No, there was never a requirement that defsystem should only be used within a .asd. Actually, the test system relies heavily on the opposite. The requirement is that .asd files be loaded in the correct context, by load-asd -- notably, the correct *package* must be bound, the correct readtable, etc.
Unfortunately, DEFSYSTEM is the only entry point we can check. So if LOAD-ASD is important, that's the only place I can check it. What else would you have me do? Check *load-truename* for "asd"?
Look, if you want to push this, then you can't object to my enforcing it. If you don't want to push this, then we should make sure DEFSYSTEM works outside the context. But I don't want to field bug reports where someone says "I tried to define this system and it didn't work," and
In the olden days, we relied on programmers to make sure that the context for DEFSYSTEM reading was appropriate. That was a pain for them sometimes, but it was clear, and it kept ASDF simple.
At some point, ASDF decided to take on the burden of establishing the context for DEFSYSTEM reading. OK, not my choice, but a reasonable decision. But I flatly refuse to maintain *both* the DWIMing in LOAD-ASD *and* DEFSYSTEM execution in arbitrary contexts. And as a programmer, I don't want ASDF to let me evaluate a DEFSYSTEM form only to beat me up because some invisible context, only apparent through reading the code, means that it doesn't work.
Pick one, DWIMing, or freestanding execution, but you get only one.
On Fri, Sep 23, 2016, 09:49 Robert Goldman <rpgoldman@sift.net mailto:rpgoldman@sift.net> wrote:
I have been working on enforcing assumptions recently added to the ASDF docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD. I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on OPERATIONs in the ASDF codebase itself. Cheers, r
On Sep 23, 2016 12:25 PM, "Robert Goldman" rpgoldman@sift.net wrote:
On 9/23/16 Sep 23 -9:52 AM, Faré wrote:
1- The requirement to always use make-operation currently only applies to code within ASDF itself and well-behaved extensions. Before you enforce it more widely, you have to make sure no one in Quicklisp does it. You could have shared initialize check that: a) no instance of that class with given initargs exists yet in *operations* and hopefully b) there initargs are always null (goodbye, make-build!)
Your docs clearly say that these MUST be built this way. If you want to say "must" then I will enforce "must." If you want "may," we can have "may" as in "For more efficient functioning, please use MAKE-OPERATION to create all operation instances."
Well, at this moment it's only a performance optimization. The goal of enforcing the invariant would be to eventually support operations with meaningful initargs. So the question is: where are you going and/or what options do you want to leave open? Enabling initargs? Forbidding them? Either way, make-operation can help you enforce the invariant. And either way requires more work. But using make-operation makes the code more future-proof.
Please articulate why you say "All operation instances MUST be created through this function"? I like the idea of having a clean API, but I also note that there is code that effectively interns the results of MAKE-OPERATION. What is the design rationale for this? Do you wish to be able to EQ-compare them? If not, why do you intern these objects in the *OPERATIONS* table? Is it really so expensive to create an OPERATION that we need to memoize? Or is it because we simply create so many, and this improves GC behavior?
Better GC is nice, but yes, having tables indexed by a pair of operation and component would be much better than having hacks like node-for... and it requires everyone using make-operation.
WRT Quicklisp: I do NOT accept that I have to check all of Quicklisp for ANYTHING. I am willing to be INFORMED by results of such checks, but I do NOT have time to make such checks.
I don't expect other people to test my SIFT code, and I don't have time to check arbitrary CL libraries, much less arbitrary CL libraries on arbitrary implementations and operating systems.
So, no. If someone else wants to test other things.
You surprise me. Historically, you've always been a strong voice for backward compatibility and being extra cautious about not breaking other people's code, especially not so without extra warnings and heads up, even more so if we're reversing something the manual used to advertise.
I wouldn't consult Quicklisp when implementing a big fix or making a backward compatible change. But when proposing a backward incompatible change, I make sure to warn all users that I can find.
2- No, there was never a requirement that defsystem should only be used within a .asd. Actually, the test system relies heavily on the opposite. The requirement is that .asd files be loaded in the correct context, by load-asd -- notably, the correct *package* must be bound, the correct readtable, etc.
Unfortunately, DEFSYSTEM is the only entry point we can check. So if LOAD-ASD is important, that's the only place I can check it. What else would you have me do? Check *load-truename* for "asd"?
Look, if you want to push this, then you can't object to my enforcing it. If you don't want to push this, then we should make sure DEFSYSTEM works outside the context. But I don't want to field bug reports where someone says "I tried to define this system and it didn't work," and
In the olden days, we relied on programmers to make sure that the context for DEFSYSTEM reading was appropriate. That was a pain for them sometimes, but it was clear, and it kept ASDF simple.
At some point, ASDF decided to take on the burden of establishing the context for DEFSYSTEM reading. OK, not my choice, but a reasonable decision. But I flatly refuse to maintain *both* the DWIMing in LOAD-ASD *and* DEFSYSTEM execution in arbitrary contexts. And as a programmer, I don't want ASDF to let me evaluate a DEFSYSTEM form only to beat me up because some invisible context, only apparent through reading the code, means that it doesn't work.
Pick one, DWIMing, or freestanding execution, but you get only one.
There too I don't understand your reasoning. Who are you helping, and who are you harming?
There always was a constraint on loading a .asd, and never was one on using defsystem. We can assume that all software that works fits those constraints, but might not fit tighter ones. Why make it harder to use defsystem? What would you check exactly?
To me, setting the syntax context is an essential service to provide to whoever writes a .asd file. The .asd maintainer cannot assume anything about the syntax context used by the user, who may not control the syntax context from the end-user. If you can't even trust the package or readtable, or character encoding, you can't even write code that has any guaranteed meaning. Note that it's called .asd rather than .lisp for a good reason: it's Lisp code, but supposed to run in a specific context.
I don't see who is served, or what feature is enabled, by introducing this backward incompatibility.
On Fri, Sep 23, 2016, 09:49 Robert Goldman <rpgoldman@sift.net mailto:rpgoldman@sift.net> wrote:
I have been working on enforcing assumptions recently added to the
ASDF
docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD. I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on
OPERATIONs
in the ASDF codebase itself. Cheers, r
On 9/23/16 Sep 23 -12:41 PM, Faré wrote:
On Sep 23, 2016 12:25 PM, "Robert Goldman" <rpgoldman@sift.net mailto:rpgoldman@sift.net> wrote:
On 9/23/16 Sep 23 -9:52 AM, Faré wrote:
1- The requirement to always use make-operation currently only applies to code within ASDF itself and well-behaved extensions. Before you enforce it more widely, you have to make sure no one in Quicklisp does it. You could have shared initialize check that: a) no instance of that class with given initargs exists yet in *operations* and hopefully b) there initargs are always null (goodbye, make-build!)
Your docs clearly say that these MUST be built this way. If you want to say "must" then I will enforce "must." If you want "may," we can have "may" as in "For more efficient functioning, please use MAKE-OPERATION to create all operation instances."
Well, at this moment it's only a performance optimization. The goal of enforcing the invariant would be to eventually support operations with meaningful initargs. So the question is: where are you going and/or what options do you want to leave open? Enabling initargs? Forbidding them? Either way, make-operation can help you enforce the invariant. And either way requires more work. But using make-operation makes the code more future-proof.
Please articulate why you say "All operation instances MUST be created through this function"? I like the idea of having a clean API, but I also note that there is code that effectively interns the results of MAKE-OPERATION. What is the design rationale for this? Do you wish to be able to EQ-compare them? If not, why do you intern these objects in the *OPERATIONS* table? Is it really so expensive to create an OPERATION that we need to memoize? Or is it because we simply create so many, and this improves GC behavior?
Better GC is nice, but yes, having tables indexed by a pair of operation and component would be much better than having hacks like node-for... and it requires everyone using make-operation.
WRT Quicklisp: I do NOT accept that I have to check all of Quicklisp for ANYTHING. I am willing to be INFORMED by results of such checks, but I do NOT have time to make such checks.
I don't expect other people to test my SIFT code, and I don't have time to check arbitrary CL libraries, much less arbitrary CL libraries on arbitrary implementations and operating systems.
So, no. If someone else wants to test other things.
You surprise me. Historically, you've always been a strong voice for backward compatibility and being extra cautious about not breaking other people's code, especially not so without extra warnings and heads up, even more so if we're reversing something the manual used to advertise.
I'm not saying that I don't want to avoid backward compatiblity issues. I just think it's too much to say "you can't change anything unless you check some large N of open source libraries by some small M of lisp implementations by some smaller P of operating systems." That's clearly impossible.
One thing about having pre-released ASDF versions is that we can run things up the flagpole and see how much we break.
But back at you -- you say you'd like to have MAKE-OPERATION be the pinch point, so that we can check initargs, exploit the fact that operations are unique and EQ-checkable, etc. But then you tell me I can't enforce the use of MAKE-OPERATION.
I don't get it. I don't see how you can have it both ways. The alternative seems to be "we're going to let you write and run code that we KNOW will break, because we don't want to signal errors." Having the *appearance* of backwards-compatibility is worse than obviously breaking backwards-compatibility.
So, please: either let's get rid of MAKE-OPERATION, kill its memoizing, and go back to letting people use MAKE-INSTANCE, or let me get on with enforcing its use.
I'm happy to see MAKE-OPERATION die (or be left around for backwards compatibility as just a shell around MAKE-INSTANCE), and reduce the code size and complexity of ASDF. Or I'm happy to enforce its use. But the midpoint is untenable.
I guess the final point is: please don't use the word "must," unless you mean "must." I feel like you're stabbing me in the back when I've tried to take your stricture seriously, and suddenly it turns out "must means may."
I wouldn't consult Quicklisp when implementing a big fix or making a backward compatible change. But when proposing a backward incompatible change, I make sure to warn all users that I can find.
2- No, there was never a requirement that defsystem should only be used within a .asd. Actually, the test system relies heavily on the opposite. The requirement is that .asd files be loaded in the correct context, by load-asd -- notably, the correct *package* must be bound, the correct readtable, etc.
Unfortunately, DEFSYSTEM is the only entry point we can check. So if LOAD-ASD is important, that's the only place I can check it. What else would you have me do? Check *load-truename* for "asd"?
Look, if you want to push this, then you can't object to my enforcing it. If you don't want to push this, then we should make sure DEFSYSTEM works outside the context. But I don't want to field bug reports where someone says "I tried to define this system and it didn't work," and
In the olden days, we relied on programmers to make sure that the context for DEFSYSTEM reading was appropriate. That was a pain for them sometimes, but it was clear, and it kept ASDF simple.
At some point, ASDF decided to take on the burden of establishing the context for DEFSYSTEM reading. OK, not my choice, but a reasonable decision. But I flatly refuse to maintain *both* the DWIMing in LOAD-ASD *and* DEFSYSTEM execution in arbitrary contexts. And as a programmer, I don't want ASDF to let me evaluate a DEFSYSTEM form only to beat me up because some invisible context, only apparent through reading the code, means that it doesn't work.
Pick one, DWIMing, or freestanding execution, but you get only one.
There too I don't understand your reasoning. Who are you helping, and who are you harming?
There always was a constraint on loading a .asd, and never was one on using defsystem. We can assume that all software that works fits those constraints, but might not fit tighter ones. Why make it harder to use defsystem? What would you check exactly?
I don't get it. You are saying on the one hand "you have to have the right context to evaluate the contents of an asd file," and on the other hand "don't stop me from evaluating ... the contents of the asd file." What is it about an ASD file reading that requires this context, if it's not the
Loading an ASD is not a lisp thing, though. Loading an ASD file involves evaluating the forms inside the file.
To me, setting the syntax context is an essential service to provide to whoever writes a .asd file. The .asd maintainer cannot assume anything about the syntax context used by the user, who may not control the syntax context from the end-user. If you can't even trust the package or readtable, or character encoding, you can't even write code that has any guaranteed meaning. Note that it's called .asd rather than .lisp for a good reason: it's Lisp code, but supposed to run in a specific context.
But that's true of EVERY Lisp file! If you just write arbitrary code, that relies on symbols from specific packages, and syntax from specific readtables, and you don't put an IN-PACKAGE and readtable specification, it will break. But I don't try to control what you do when you are writing arbitrary lisp code. So clearly it's not "essential." Why do we do it for ASD files? What's so hard about leaving it to the programmer to get this right? Why did we have to make that our job instead of the programmer's job? Worse, we have broken that invaluable debugging tool of interactive execution of code, by encouraging programmers to rely on our use of LOAD-ASD.
Actually, when I read your reasoning, I think I see that this *is* generally true (especially your remarks about the readtable). This suggests to me that what you are really doing here is chewing off a tiny corner of CL, and in that tiny corner, you are trying to fix a thing that is broken about the language -- but only in that tiny corner.
I concede that this ship has sailed, and there's no tearing down LOAD-ASD, although I dearly wish I could.
But again, I feel very frustrated, because I feel like you are saying to me "there's SOMETHING in every .asd file that should only be evaluated in the context of LOAD-ASD, but I'm telling you it's not DEFSYSTEM, and I'm not telling you what it is." Or even, "asd files should never be loaded, or evaluated, except through LOAD-ASD, but any form in an asd file can be."
So, ok, what IS it about an ASD file, that is not DEFSYSTEM, that must be evaluated inside LOAD-ASD? If it's just "you could wreck the syntax behind my back," yeah, that's true, but you could wreck the syntax behind my back and wreck the library I'm loading, too, so why is it ASDF's job to fix that here? If LOAD is broken, why is it our job to make LOAD-ASD and fix a tiny corner of the problem?
I don't see who is served, or what feature is enabled, by introducing this backward incompatibility.
On Fri, Sep 23, 2016, 09:49 Robert Goldman <rpgoldman@sift.net
<mailto:rpgoldman@sift.net mailto:rpgoldman@sift.net>> wrote:
I have been working on enforcing assumptions recently added tothe ASDF
docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD. I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE onOPERATIONs
in the ASDF codebase itself. Cheers, r
On Fri, 2016-09-23 at 13:55 -0500, Robert Goldman wrote: [...]
You surprise me. Historically, you've always been a strong voice for backward compatibility and being extra cautious about not breaking other people's code, especially not so without extra warnings and heads up, even more so if we're reversing something the manual used to advertise.
I'm not saying that I don't want to avoid backward compatiblity issues. I just think it's too much to say "you can't change anything unless you check some large N of open source libraries by some small M of lisp implementations by some smaller P of operating systems." That's clearly impossible.
You should change what you want, while being aware that if you break people's stuff they will complain and in the worse case reject the change altogether, so when when considering multiple paths for implementing a new feature and deprecating old ones, it's useful to know how much code each of the alternatives will break.
This is were testing Quicklisp (or a small but important subset thereof) comes useful; and if one breaks the olden ways, briefly documenting how to convert to the new ways is polite (in the old meaning, having the same root as "politics").
One thing about having pre-released ASDF versions is that we can run things up the flagpole and see how much we break.
But back at you -- you say you'd like to have MAKE-OPERATION be the pinch point, so that we can check initargs, exploit the fact that operations are unique and EQ-checkable, etc. But then you tell me I can't enforce the use of MAKE-OPERATION.
I don't get it. I don't see how you can have it both ways. The alternative seems to be "we're going to let you write and run code that we KNOW will break, because we don't want to signal errors." Having the *appearance* of backwards-compatibility is worse than obviously breaking backwards-compatibility.
So, please: either let's get rid of MAKE-OPERATION, kill its memoizing, and go back to letting people use MAKE-INSTANCE, or let me get on with enforcing its use.
I'm happy to see MAKE-OPERATION die (or be left around for backwards compatibility as just a shell around MAKE-INSTANCE), and reduce the code size and complexity of ASDF. Or I'm happy to enforce its use. But the midpoint is untenable.
I guess the final point is: please don't use the word "must," unless you mean "must." I feel like you're stabbing me in the back when I've tried to take your stricture seriously, and suddenly it turns out "must means may."
I wouldn't consult Quicklisp when implementing a big fix or making a backward compatible change. But when proposing a backward incompatible change, I make sure to warn all users that I can find.
2- No, there was never a requirement that defsystem should only be used within a .asd. Actually, the test system relies heavily on the opposite. The requirement is that .asd files be loaded in the correct context, by load-asd -- notably, the correct *package* must be bound, the correct readtable, etc.
Unfortunately, DEFSYSTEM is the only entry point we can check. So if LOAD-ASD is important, that's the only place I can check it. What else would you have me do? Check *load-truename* for "asd"?
Look, if you want to push this, then you can't object to my enforcing it. If you don't want to push this, then we should make sure DEFSYSTEM works outside the context. But I don't want to field bug reports where someone says "I tried to define this system and it didn't work," and
In the olden days, we relied on programmers to make sure that the context for DEFSYSTEM reading was appropriate. That was a pain for them sometimes, but it was clear, and it kept ASDF simple.
At some point, ASDF decided to take on the burden of establishing the context for DEFSYSTEM reading. OK, not my choice, but a reasonable decision. But I flatly refuse to maintain *both* the DWIMing in LOAD-ASD *and* DEFSYSTEM execution in arbitrary contexts. And as a programmer, I don't want ASDF to let me evaluate a DEFSYSTEM form only to beat me up because some invisible context, only apparent through reading the code, means that it doesn't work.
Pick one, DWIMing, or freestanding execution, but you get only one.
There too I don't understand your reasoning. Who are you helping, and who are you harming?
There always was a constraint on loading a .asd, and never was one on using defsystem. We can assume that all software that works fits those constraints, but might not fit tighter ones. Why make it harder to use defsystem? What would you check exactly?
I don't get it. You are saying on the one hand "you have to have the right context to evaluate the contents of an asd file," and on the other hand "don't stop me from evaluating ... the contents of the asd file." What is it about an ASD file reading that requires this context, if it's not the
Loading an ASD is not a lisp thing, though. Loading an ASD file involves evaluating the forms inside the file.
To me, setting the syntax context is an essential service to provide to whoever writes a .asd file. The .asd maintainer cannot assume anything about the syntax context used by the user, who may not control the syntax context from the end-user. If you can't even trust the package or readtable, or character encoding, you can't even write code that has any guaranteed meaning. Note that it's called .asd rather than .lisp for a good reason: it's Lisp code, but supposed to run in a specific context.
But that's true of EVERY Lisp file! If you just write arbitrary code, that relies on symbols from specific packages, and syntax from specific readtables, and you don't put an IN-PACKAGE and readtable specification, it will break. But I don't try to control what you do when you are writing arbitrary lisp code. So clearly it's not "essential." Why do we do it for ASD files? What's so hard about leaving it to the programmer to get this right? Why did we have to make that our job instead of the programmer's job?
Because most Lispers don't know ASDF enough to get it right. If it can be fixed in a single point, it should.
Worse, we have broken that invaluable debugging tool of interactive execution of code, by encouraging programmers to rely on our use of LOAD-ASD.
LOAD-ASD is still interactive execution of code, just in a saf(er) implicit environment. It's not a distinct parser.
Actually, when I read your reasoning, I think I see that this *is* generally true (especially your remarks about the readtable). This suggests to me that what you are really doing here is chewing off a tiny corner of CL, and in that tiny corner, you are trying to fix a thing that is broken about the language -- but only in that tiny corner.
I concede that this ship has sailed, and there's no tearing down LOAD-ASD, although I dearly wish I could.
But again, I feel very frustrated, because I feel like you are saying to me "there's SOMETHING in every .asd file that should only be evaluated in the context of LOAD-ASD, but I'm telling you it's not DEFSYSTEM, and I'm not telling you what it is." Or even, "asd files should never be loaded, or evaluated, except through LOAD-ASD, but any form in an asd file can be."
So, ok, what IS it about an ASD file, that is not DEFSYSTEM, that must be evaluated inside LOAD-ASD? If it's just "you could wreck the syntax behind my back," yeah, that's true, but you could wreck the syntax behind my back and wreck the library I'm loading, too, so why is it ASDF's job to fix that here? If LOAD is broken, why is it our job to make LOAD-ASD and fix a tiny corner of the problem?
Because unless you have the resources to fix the language in a major way, you can only do small incremental improvements. Very pragmatic. Not ideal.
On 9/23/16 Sep 23 -6:24 PM, Stelian Ionescu wrote:
So, ok, what IS it about an ASD file, that is not DEFSYSTEM, that
must be evaluated inside LOAD-ASD? If it's just "you could wreck the syntax behind my back," yeah, that's true, but you could wreck the syntax behind my back and wreck the library I'm loading, too, so why is it ASDF's job to fix that here? If LOAD is broken, why is it our job to make LOAD-ASD and fix a tiny corner of the problem?
Because unless you have the resources to fix the language in a major way, you can only do small incremental improvements. Very pragmatic. Not ideal.
But my claim is that it is NOT an improvement if you turn a language which is incrementally compilable into a language that only works if you load it through LOAD-ASD.
What we have done with LOAD-ASD is to take away code that is runnable with compile-sexp and replace it with code that is only runnable with LOAD-ASD.
It's obviously exaggerating, but it's a step towards taking what makes CL great, and taking that away and giving us C instead.
Now, of course, people like me can put our packages in our asd files and things should keep working, but what's the contract with the programmer here?
Are we going to shove more stuff into LOAD-ASD, further divorcing the ASD-dialect from mainstream CL?
Anyway, here's my proposed solution:
1. Don't enforce the use of LOAD-ASD. Probably people can understand this well enough.
2. Do enforce the use of MAKE-OPERATION. I doubt that this will interfere with people that badly (particularly because most people won't be doing it), and if it does, it's not that hard to replace a MAKE-INSTANCE with a MAKE-OPERATION. Also, allowing people to willy-nilly create instances of a class whose instances are supposed to be interned (i.e., EQ-comparable) is a Bad Thing.
If CL-TEST-GRID or some other testing shows that this causes Quicklisp to melt-down, then we can consider replacing my continuable error call with a WARNING.
ACCEPTABLE-P?
On Fri, 2016-09-23 at 18:32 -0500, Robert Goldman wrote:
On 9/23/16 Sep 23 -6:24 PM, Stelian Ionescu wrote:
So, ok, what IS it about an ASD file, that is not DEFSYSTEM, that
must be evaluated inside LOAD-ASD? If it's just "you could wreck the syntax behind my back," yeah, that's true, but you could wreck the syntax behind my back and wreck the library I'm loading, too, so why is it ASDF's job to fix that here? If LOAD is broken, why is it our job to make LOAD-ASD and fix a tiny corner of the problem?
Because unless you have the resources to fix the language in a major way, you can only do small incremental improvements. Very pragmatic. Not ideal.
But my claim is that it is NOT an improvement if you turn a language which is incrementally compilable into a language that only works if you load it through LOAD-ASD.
What we have done with LOAD-ASD is to take away code that is runnable with compile-sexp and replace it with code that is only runnable with LOAD-ASD.
It's obviously exaggerating, but it's a step towards taking what makes CL great, and taking that away and giving us C instead.
Now, of course, people like me can put our packages in our asd files and things should keep working, but what's the contract with the programmer here?
Are we going to shove more stuff into LOAD-ASD, further divorcing the ASD-dialect from mainstream CL?
Anyway, here's my proposed solution:
- Don't enforce the use of LOAD-ASD. Probably people can understand
this well enough.
Enforce it but add a way to disable it, but not a simple dynamic variable. A new function, e.g.
(setf (asdf:flag :enforce-load-asd) nil)
And a way for the programmer to query the current set of ASDF flags.
On Fri, Sep 23, 2016 at 7:32 PM, Robert Goldman rpgoldman@sift.net wrote:
But my claim is that it is NOT an improvement if you turn a language which is incrementally compilable into a language that only works if you load it through LOAD-ASD.
It works perfectly in slime, if you enable swank-asdf.
What we have done with LOAD-ASD is to take away code that is runnable with compile-sexp and replace it with code that is only runnable with LOAD-ASD.
It was never more "runnable" than it is. Never. ASDF 1 created a temporary package around loading a .asd file, and changed *package* to that. ASDF 3 bailed on that and introduced ASDF-USER, there again binding *package* to that. There was no regression at any point. Actually, because the package used to be destroyed, ASDF 1 was quite a pain to trace or debug code written without in-package, whereas ASDF 3 makes it easier.
Users have always been able to write a .asd file without having to write in-package, and writing in-package would NOT protect you against someone loading a system while under a package that re-bound in-package (e.g. some CL-in-CL implementation).
If you fail to bind proper syntax around load .asd's, then you prevent people from using load-system from within the middle of their arbitrary session where they changed the syntax. Worse, you prevent them from ever using any function that indirectly does, which means they have to worry about the implementation details of all their transitive dependencies, which is absurd and contrary to the need-to-know principle.
Once again, swank-asdf also knows how to load a .asd file with load-asd. It might not be good at identifying the package as asdf-user, but then that's a bug in slime that can be fixed in slime.
It's obviously exaggerating, but it's a step towards taking what makes CL great, and taking that away and giving us C instead.
I beg to differ. You want to force people who write .asd files to be super defensive and use in-package, for fewer actual warranties, with a major compatibility break, for no tangible benefit whatsoever to anyone.
Now, of course, people like me can put our packages in our asd files and things should keep working, but what's the contract with the programmer here?
There was a clear contract: the .asd writer doesn't have to care about the context of the user (that he doesn't control), and the asdf user doesn't have to care about the context of the .asd file (that gets automatically setup for him). You're breaking this contract, and
Are we going to shove more stuff into LOAD-ASD, further divorcing the ASD-dialect from mainstream CL?
It's not divorcing the asd dialect from CL. It's setting up the syntactic context so to isolate the .asd writer from the .asd user, as is the proper thing to do.
Anyway, here's my proposed solution:
- Don't enforce the use of LOAD-ASD. Probably people can understand
this well enough.
- Do enforce the use of MAKE-OPERATION. I doubt that this will
interfere with people that badly (particularly because most people won't be doing it), and if it does, it's not that hard to replace a MAKE-INSTANCE with a MAKE-OPERATION. Also, allowing people to willy-nilly create instances of a class whose instances are supposed to be interned (i.e., EQ-comparable) is a Bad Thing.
To be clear, I'm all in favor of enforcing the use of MAKE-OPERATION. I'm just somewhat opposed about doing without providing heads up to users. If you're not willing to hunt users in Quicklisp (for which Anton V's cl-test-grid can help), then it's better to give them heads up to find out. e.g. document things loadly in the manual (that still talks about make-instance), announce with the next release, and enforce the following release.
If CL-TEST-GRID or some other testing shows that this causes Quicklisp to melt-down, then we can consider replacing my continuable error call with a WARNING.
ACCEPTABLE-P?
Yes.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Dost thou love life? Then do not squander time, for that's the stuff life is made of. — Benjamin Franklin
I have been working on enforcing assumptions recently added to the ASDF docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD.
I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on OPERATIONs in the ASDF codebase itself.
Thank you for the work.
-
Faré -
Robert Goldman -
Stelian Ionescu