On Fri, 2016-09-23 at 13:55 -0500, Robert Goldman wrote: [...]
You surprise me. Historically, you've always been a strong voice for backward compatibility and being extra cautious about not breaking other people's code, especially not so without extra warnings and heads up, even more so if we're reversing something the manual used to advertise.
I'm not saying that I don't want to avoid backward compatiblity issues. I just think it's too much to say "you can't change anything unless you check some large N of open source libraries by some small M of lisp implementations by some smaller P of operating systems." That's clearly impossible.
You should change what you want, while being aware that if you break people's stuff they will complain and in the worse case reject the change altogether, so when when considering multiple paths for implementing a new feature and deprecating old ones, it's useful to know how much code each of the alternatives will break.
This is were testing Quicklisp (or a small but important subset thereof) comes useful; and if one breaks the olden ways, briefly documenting how to convert to the new ways is polite (in the old meaning, having the same root as "politics").
One thing about having pre-released ASDF versions is that we can run things up the flagpole and see how much we break.
But back at you -- you say you'd like to have MAKE-OPERATION be the pinch point, so that we can check initargs, exploit the fact that operations are unique and EQ-checkable, etc. But then you tell me I can't enforce the use of MAKE-OPERATION.
I don't get it. I don't see how you can have it both ways. The alternative seems to be "we're going to let you write and run code that we KNOW will break, because we don't want to signal errors." Having the *appearance* of backwards-compatibility is worse than obviously breaking backwards-compatibility.
So, please: either let's get rid of MAKE-OPERATION, kill its memoizing, and go back to letting people use MAKE-INSTANCE, or let me get on with enforcing its use.
I'm happy to see MAKE-OPERATION die (or be left around for backwards compatibility as just a shell around MAKE-INSTANCE), and reduce the code size and complexity of ASDF. Or I'm happy to enforce its use. But the midpoint is untenable.
I guess the final point is: please don't use the word "must," unless you mean "must." I feel like you're stabbing me in the back when I've tried to take your stricture seriously, and suddenly it turns out "must means may."
I wouldn't consult Quicklisp when implementing a big fix or making a backward compatible change. But when proposing a backward incompatible change, I make sure to warn all users that I can find.
2- No, there was never a requirement that defsystem should only be used within a .asd. Actually, the test system relies heavily on the opposite. The requirement is that .asd files be loaded in the correct context, by load-asd -- notably, the correct *package* must be bound, the correct readtable, etc.
Unfortunately, DEFSYSTEM is the only entry point we can check. So if LOAD-ASD is important, that's the only place I can check it. What else would you have me do? Check *load-truename* for "asd"?
Look, if you want to push this, then you can't object to my enforcing it. If you don't want to push this, then we should make sure DEFSYSTEM works outside the context. But I don't want to field bug reports where someone says "I tried to define this system and it didn't work," and
In the olden days, we relied on programmers to make sure that the context for DEFSYSTEM reading was appropriate. That was a pain for them sometimes, but it was clear, and it kept ASDF simple.
At some point, ASDF decided to take on the burden of establishing the context for DEFSYSTEM reading. OK, not my choice, but a reasonable decision. But I flatly refuse to maintain *both* the DWIMing in LOAD-ASD *and* DEFSYSTEM execution in arbitrary contexts. And as a programmer, I don't want ASDF to let me evaluate a DEFSYSTEM form only to beat me up because some invisible context, only apparent through reading the code, means that it doesn't work.
Pick one, DWIMing, or freestanding execution, but you get only one.
There too I don't understand your reasoning. Who are you helping, and who are you harming?
There always was a constraint on loading a .asd, and never was one on using defsystem. We can assume that all software that works fits those constraints, but might not fit tighter ones. Why make it harder to use defsystem? What would you check exactly?
I don't get it. You are saying on the one hand "you have to have the right context to evaluate the contents of an asd file," and on the other hand "don't stop me from evaluating ... the contents of the asd file." What is it about an ASD file reading that requires this context, if it's not the
Loading an ASD is not a lisp thing, though. Loading an ASD file involves evaluating the forms inside the file.
To me, setting the syntax context is an essential service to provide to whoever writes a .asd file. The .asd maintainer cannot assume anything about the syntax context used by the user, who may not control the syntax context from the end-user. If you can't even trust the package or readtable, or character encoding, you can't even write code that has any guaranteed meaning. Note that it's called .asd rather than .lisp for a good reason: it's Lisp code, but supposed to run in a specific context.
But that's true of EVERY Lisp file! If you just write arbitrary code, that relies on symbols from specific packages, and syntax from specific readtables, and you don't put an IN-PACKAGE and readtable specification, it will break. But I don't try to control what you do when you are writing arbitrary lisp code. So clearly it's not "essential." Why do we do it for ASD files? What's so hard about leaving it to the programmer to get this right? Why did we have to make that our job instead of the programmer's job?
Because most Lispers don't know ASDF enough to get it right. If it can be fixed in a single point, it should.
Worse, we have broken that invaluable debugging tool of interactive execution of code, by encouraging programmers to rely on our use of LOAD-ASD.
LOAD-ASD is still interactive execution of code, just in a saf(er) implicit environment. It's not a distinct parser.
Actually, when I read your reasoning, I think I see that this *is* generally true (especially your remarks about the readtable). This suggests to me that what you are really doing here is chewing off a tiny corner of CL, and in that tiny corner, you are trying to fix a thing that is broken about the language -- but only in that tiny corner.
I concede that this ship has sailed, and there's no tearing down LOAD-ASD, although I dearly wish I could.
But again, I feel very frustrated, because I feel like you are saying to me "there's SOMETHING in every .asd file that should only be evaluated in the context of LOAD-ASD, but I'm telling you it's not DEFSYSTEM, and I'm not telling you what it is." Or even, "asd files should never be loaded, or evaluated, except through LOAD-ASD, but any form in an asd file can be."
So, ok, what IS it about an ASD file, that is not DEFSYSTEM, that must be evaluated inside LOAD-ASD? If it's just "you could wreck the syntax behind my back," yeah, that's true, but you could wreck the syntax behind my back and wreck the library I'm loading, too, so why is it ASDF's job to fix that here? If LOAD is broken, why is it our job to make LOAD-ASD and fix a tiny corner of the problem?
Because unless you have the resources to fix the language in a major way, you can only do small incremental improvements. Very pragmatic. Not ideal.