On Sun, Jan 12, 2014 at 10:50 PM, Robert P. Goldman rpgoldman@sift.info wrote:
The best we can do is issue a warning whenever code defines a subclass of OPERATION, and allow the user to indicate that s/he understands and wishes to proceed.
I agree in principle, and I believe that would have been a great thing for me to do last year. But I failed to do it, and I fear it's too late now: I strongly suspect the remaining population of people who wrote subclasses of ASDF operations yet didn't get their code fixed is now empty.
That said, though I believe it's too late to do anything about the change in semantics of OPERATION, I think it's a good idea to provide a mechanism that will allow graceful upgrade in the future.
This is a very nasty case, since the user's agreement is of necessity file-scoped.
We could make it defsystem-scoped, by having defsystem specify an :asdf-version "3.1" or some such. Of course, since that will break any older version of ASDF, this will need be #+asdf3.1 :asdf-version #+asdf3.1 "3.1" until 3.1 becomes universal, which will take a year or two. But if you don't add this option NOW, then you'll be in trouble a year or two from now when you need it.
Now of course, since an OPERATION class is not directly linked to a system, that mightn't help in this case.
The best thing I can think of is to have a wrapper around ASDF system loading that will establish a binding for a variable that the user can rebind local to a particular file.
That will allow the user to modify individual ASDF system definitions to indicate that this is ok.
That sounds particularly messy to me. I admit that the fewer special variables, the happier I am. Especially variables that modify the semantics, and that get superdupermessy if something recursively loads a .asd file, or worse does it without going through load-asd.
That's not a very good solution, actually, since the OPERATION subclass might be defined outside an ASD file, in a normal .lisp file. So perhaps wrapping COMPILE-OP and LOAD-OP is also necessary.
It's even worse, because OPERATION subclasses are often defined in a good old Lisp file, called asdf-support.lisp or some such. A flag around the loading of asd files won't help. Or now, you're going to have to do checks around every file.
Frankly, I believe this has a huge cost for negligible benefit after the fact. As for how to deal with future incompatible changes, it's a more interesting question, and I have no great answer, but that flag idea sounds like an especially bad solution.
I'd like to get this right before going forward, but I see only a choice of unpleasant alternatives.
If anyone has a better one, I'm open to suggestions.
I don't see any alternative that can fix the past. Making things future-proof is more interesting. Back in the ASDF2 days, and then again with ASDF3 and ASDF3.1, I had to define and refine various hooks for hot upgrade of ASDF, after finding the hard way where they were lacking. Other hooks may have to be defined for systems and for extensions to specify what version of ASDF they rely upon. Since these hooks don't exist yet, I believe it's erroneous to act as if they used to exist. But it's still time to define them.
Unfortunately, to take your point about ASDF needing to let the person who knows provide the knowledge, the only person who knows whether an OPERATION redefinition is good is the system definer, and NOT the poor programmer who loads a library.
Yes. And unless you come up with something extremely clever, I don't see any way to distinguish between libraries designed for ASDF2 and libraries designed for ASDF3 through programmatic means in a way that doesn't retrospectively break plenty of packages.
So I'm inclined to do what's generally bad practice and explicitly provide an out for all of the quicklisp systems that you have checked. Only, oh dear, most people are not providing :VERSION attributes, which means that we cannot distinguish patched from unpatched libraries....
My recommendation is to provide and all out for all existing systems, and not worry about it for the past anymore. I'll bet you $50 that no one is going to complain next year about a broken OPERATION that isn't already in quicklisp (and thus hopefully already fixed), and wasn't written or maintained by one of the people who's subscribed to this list.
Do you know: can we rely on :VERSION information to identify Quicklisp libraries that you have patched? If not, could we make that happen?
I'm not sure what you mean. Not every library has a usable :VERSION. Quicklisp probably maintains some date information, but according to my recent audit, everything in Quicklisp seems to be fine already, so it's just a matter of telling people "update to the latest version of quicklisp and/or your library".
With all due respect, I think you are being a little unfair to the the victims in this case.
Well, the entire community was dysfunctional with respect to ASDF. Those who extended a buggy ASDF that nobody understood (and they couldn't possibly, because it didn't make sense) have their small share of responsibility in the mess, and are paying the price now. Considering how much effort it was for me to fix ASDF, that was probably a wise choice of theirs; but the price of living with the bugs is still rightfully theirs to pay.
In the past, if you assumed that subclassing OPERATION would give you the same dependency behavior as LOAD-OP, then you weren't far wrong.
But now you get no dependency propagation, and that's almost always wrong.
I disagree. You could rely on the same dependency behavior as LOAD-OP, but that behavior was wrong for LOAD-OP, and usually wronger for whichever operation you were implementing, too. I haven't seen a single operation for which the ASDF1 algorithm for propagating dependencies is correct, and I'm convinced such beast can't exist.
There are a few cases where you want sideway propagation (lib-op); a few cases where you might want downward propagation (docstring-op ?); but sideway and downward without an upward propagation to ensure order, I can hardly conceive it: if you care about order, you almost certainly need an upward preparation operation, and if you don't you probably don't need sideway.
I am not saying there was a better solution, but I don't think it's fair to say that people who subclassed OPERATION were relying on bugs. There WERE bugs in the behavior they relied upon, but that's not the same as relying on bugs....
But this is a quibble. I'm not saying there was a better solution; I'm just saying that fixing these bugs is not without costs, and it's not fair to blame the people who had to live with buggy libraries.
As the person who paid the price to fix things, I think it's fair to say those who didn't should pay the price of living with bugs; I'm not saying they made the wrong choice; I'm saying paying this price is part of their choice; let them assume the bitter part as well as the sweet part (which was being able to make do cheaply without addressing the deep issue).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Economic illiteracy often leads one to take for wealth creation or cost reduction what is only a forced displacement of activity, with no primary gain, and a lot of secondary costs and negative side-effects.