On Sat, Mar 15, 2014 at 4:52 PM, Robert P. Goldman rpgoldman@sift.info wrote:
Here is an example of a kind of system that will be broken by this proposed change:
We have a distributed system that is lisp-based.
There is a small set of ASDF systems that constitutes the common base
of the modules of this distributed system.
- For each module, there is an ASDF system, which incorporates the
common base, the module proper, and possibly some additional systems.
The common base has a readtable (and it may have a PPRINT-DISPATCH-TABLE).
The distributed modules systems assume, safely, that the readtable
will be set by the common base.
- Code in the common base uses readtable-setting functions with the
default values for the readtable.
The clean thing to do would be to use named-readtables and/or cl-syntax, and have each file evaluate (in-readtable :foo) or have a perform :around method or around-compile hook that does it for you.
We could also support your doing (in-readtable :foo) only once for the entire system, though it's trickier.
(Question, either way: how does the readtable get re-initialized (or not) when you recompile a modified system? Should you start from a fresh readtable every time, or reuse the readtable of the end of compilation while at the start of compilation?)
But polluting arbitrary systems loaded after you is definitely a bad idea: if someone else write a system that depends on part of yours, that also depends on unrelated systems, these unrelated systems will end up being affected by your readtable changes, and, if they are just as "dirty" as yours, they might in turn affect your system.
Such idiom is guaranteed to break, badly, in *some* circumstances, and we shouldn't encourage or support it. The readtable interferences will only get worse at the REPL, when a modified system is re-compiled with readtable modifications from a system that it doesn't depend on, and the next time you try to load it from clean, you find it mysteriously depends on the other system.
As I understand it, the proposed change will break this kind of system. Indeed, I think it's likely to break the parent common base system as well as breaking the system composition (the use of the readtable is not confined to a single file in the common base, and I believe it straddles multiple subsystems, which are represented as individual ASDF systems).
Yes, it's a feature to break this kind of system, so as to enable instead systems where the programmer can safely rely on his system not being either victim or author of interference with other systems, and may both rely on a sane readtable and be able to modify his readtable safely.
But that there are systems like that suggests that the change should not be made without a lot of heads up, and so definitely not for 3.1.1.
I'm not going to defend the structure of this code, but I strongly feel that it is not the job of ASDF to impose a personal opinion that this coding pattern is bad, and should be *forced to break* when previously it worked perfectly adequately.
It did not work "perfectly". It happened to work because you control the build process to make sure you don't interfere with other systems, and ASDF de facto imposes on general purpose libraries that they can't safely do anything fancy with readtables, which sucks.
To allow libraries at large to safely enjoy localized syntax modification at the cost of requiring a little bit of discipline from a handful of dirty systems sounds like a pretty good deal to me, and totally the job of ASDF. Without ASDF, there's no way the libraries will be safe by default. Once again, the example of a random load-system at the REPL causing havoc because I had syntax modifications at the REPL is pretty dreadful.
To the best of my knowledge, the readtable manipulation functions in CL don't provide any easy replacement for the pattern of coding applied here.
named-readtables and cl-syntax are there to help. ASDF could and in my opinion should make things safe by default instead of unsafe by default.
Yes, one *can* set a global to the special readtable, and then one can write a special PERFORM method (one? or many?) that will bind *READTABLE* to the value of this global.
Yes, and many systems do that, including ironclad. Actually, inspecting the ironclad failure in cl-test-grid, I find that the problem was one with my code, whereby my :around method was run inside ironclad's :around method rather than outside, causing the breakage. I will implement a better protocol, and see how much breakage that causes.
This is already a monumental PITA, since the PERFORM method will have to refer to a variable named in a package that doesn't exist at the time the ASDF system is defined. It also forces our poor programmer to know more than s/he ever wanted to know about ASDF, and IMO inappropriately bleeds details of the code of the system into the system's build file.
The referring to the variable can be done through the now usual "read the string late" method as long used to name component classes and functions and recently operation classes.
I don't see how this "bleeds details about ASDF". Hacking readtables was always tricky. Up until now, it is unsafe to modify the global readtable, and for that reason, library authors are strongly recommended to not do it, while application authors do it at their own risk. With some help from ASDF, it can be made safe for everyone, and those application authors who used to do it the ugly way can still do it at the expense of one call to in-readtable or similar.
I believe that the proposed hygiene practice *also* leaves the programmer in an uncomfortable position if s/he incrementally compiles functions inside an editor buffer, since the readtable will not be properly set. I.e., AFAICT, although careful binding of the readtable, rather than treating it as a global variable, feels like the Right Thing, it has undesirable consequences because we don't have good ways to bind the readtable around the editor. If a -*- READTABLE: foo; -*- file header worked, this would be less of a concern, but using a header like that requires named readtables.
No, it's the CURRENT practice that makes it unsafe for everyone to compile files, because that will be done with the *current* readtable, which will be completely inappropriate for files in a different system. I've been bitten, hard, by that.
With my proposed changes (see what I pushed in branch standard-syntax), programmers can recompile a file and that will happen automatically in the correct readtable. (Though they will need a properly updated swank-asdf, if using SLIME.)
My conclusion is that enforcing some sort of compartmentalization of readtable is not only an inappropriate imposition of a particular style, but *also* means we are hanging the poor programmer out to dry. CL simply doesn't provide good tools for manipulating the readtable in the way we are proposing to enforce.
IMO, named readtables are the way to go, but they are not a common feature of the language, and the portable library is bit-rotted.*
I will not be pushing this proposed modification into the next release, and it will take a lot of convincing for me ever to incorporate it into ASDF.
Yes, CL does provide the tools, we were just not using them. and *that* was hanging the poor programmer out to dry. By implementing per-system syntax, we're providing them the hygiene they sorely need. People who were previously leaking readtable state will have it private to their system; if they were leaking readtable state from one system to the next, that won't work anymore, but that was already a hazard to systems they were not writing, so we're trading a "that's not working for another poor sod who can't know and can't do anything about it" to a "that's not working for someone doing something ugly who should know better and can do something about it". And that was the guiding principle of ASDF2: "he who knows is who specifies the information, he who doesn't know doesn't have to do anything".
*This is partly my fault, but the current disarray of cl.net, the library's position as part of an abortive EDITOR-HINTS master system, and the previous maintainer's putting it into a revision control system (darcs) that I don't use and don't understand are additional contributing factors.
If you're the new maintainer, I suggest converting the darcs repository to git and maybe putting the library on github.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The aim of public education is not to spread enlightenment at all; it is simply to reduce as many individuals as possible to the same safe level, to breed and train a standardized citizenry, to put down dissent and originality. ― H.L. Mencken