On Wed, Apr 2, 2014 at 9:06 PM, Robert P. Goldman rpgoldman@sift.info wrote:

Faré wrote:

...

compile-file and load already bind *readtable*, which means that for asdf itself to bind *readtable* should be a no-op in the common case, and a BIG save for those who want to switch the readtable at the REPL.

I'm puzzled by this claim. As actually implemented, this seems to me to tend to *force* procedural interaction with the *READTABLE*, in ways that you deplore. But perhaps I misunderstand your proposed revision.

Well, yes, and that's what you said I had to preserve backward compatibility with, isn't it?

This branch 1- *documents* the previously implicit manual hygiene requirements for shared readtable usage 2- *enforces* that the same readtable supporting CL + shared extensions is used for every build, independently from what the user uses at the REPL, which now can be completely different.

The hygiene requirements can't be enforced without breaking backward compatibility. But we already know that in practice, people already make do, and now there is a documented guideline.To enable a read-only readtable (on SBCL, CCL and other supported implementations), use in the syntax-control branch: (setf asdf:*cl-reading-hook* 'uiop:call-with-standard-io-syntax)

If the programmer wants his/her changes to the readtable to persist over a session, in the absence of an easy way to switch readtables (e.g., by named-readtables and a READTABLE: head-of-file directive as there is in ACL's ELI, and I believe there was in Zmacs), there isn't a pleasant alternative to destructively modifying the value of *READTABLE* that will be carried over multiple calls to COMPILE-FILE and LOAD.

Yes, and this will work with my branch. Indeed, since *READTABLE* is bound by COMPILE-FILE and LOAD, any such escaping change is a side-effect to the existing *readtable* datastructure as supplied by ASDF, and not due to a new datastructure being bound to *READTABLE*.

Therefore, all ASDF has to do is remember the readtable that was used while loading ASDF itself, and make sure it is always reused while compiling CL code. Unless you switch the *READTABLE*, which you can't do from within an ASDF system (at least not without advanced tricks), the whole thing is thus a nop. And from those who *do* switch the *READTABLE* at the REPL, which supposes incompatibility, we really do not want to use such incompatible readtable, and do want the readtable bound back to normal when re-compiling.

Those who *really* want to trick ASDF into using an incompatible readtable will have to reconfigure ASDF to store output files somewhere else, anyway, and binding the asdf:*shared-readtable* will be a minor detail along the way — modifying output-translations would probably be important there.

But my understanding of your proposal is that you will somehow bind the readtable to a *clean* readtable, rather than the current readtable, so the behavior you propose will be radically different from the behavior specified by the standard. Is that not so?

Not so much *clean* than *shared*. Or if "clean", staying so by every programmer respecting the now documented hygiene discipline.

Consider the case where there's no ASDF (as at the time of the CL standardization), and no named-readtables. What's the programmer to do aside from picking a readtable and mangling it? I suppose the super-wizard could switch back and forth, but nothing in the spec seems to make this at all pleasant.

Well, I've been there, with Scribble, Exscribe, and fare-quasiquote. In the beginning, indeed, I modified the global readtable. Then I learned good practices, and instead stored my readtable in a variable, and did an (eval-when (:compile-toplevel :execute) (setf *readtable* *my-readtable*)) in every file that needed it. These days, I just use named-readtables for the same general effect. It's not super-pleasant indeed, but it works... except that if I do it at the REPL then use load-system, it can pollute my output translation cache, and then I have to remove it.

In practice, my experience is that if, e.g., you were working in Nisp, you just switched to the Nisp readtable for your whole session, and that was it. So the Nisp system definition would copy the readtable, change *READTABLE* to point to the copy, and then following systems (which would be written in Nisp, rather than vanilla CL), would get the readtable in a smooth way.

No, you can't change the binding of *READTABLE* in a LOAD or COMPILE-FILE, because CL binds it around these two functions. You must either side-effect the shared readable, or use eval-when to select another one. Or with ASDF, you can have a perform around wrapper select a different readtable before you load.

With my patch, there are better entry points for such wrappers, and a default wrapper that ensures all builds get the same shared CL readtable that is not any of the incompatible readtable, in particular not the fare-quasiquote readtable.

Also, COMPILE-FILE binds the readtable to the *current value* of the readtable. This lets us copy it if we want, but given that the behavior of SET-MACRO-CHARACTER is to destructively modify its readtable argument (which may be *READTABLE*), I don't see that this gives any precedent to the proposed syntax hygiene.

If COMPILE-FILE bound *READTABLE* to a *COPY* of the current readtable, then I would see the precedent, but as it is, I'm still not convinced.

The "hygiene" for the shared readtable is a matter of following these conventions, which are the current docstring for uiop:*shared-readtable*:

This shared readtable allows legacy applications to keep modifying a global shared readtable while maintaining some hygiene for those who want to use their own readtable. It is subject to the following restrictions, which always existed but were previously implicit: A- no modifying any standard character, B- no two dependencies assigning different meaning to the same non-standard character. Using any non-standard character while expecting the implementation to treat it some way counts as such an assignment of meaning. C- libraries need to document these assignments of meaning to non-standard characters. D- free software libraries will register these changes on: http://www.cliki.net/Macro%20Characters

Indeed, I am only more convinced that the proposed behavior is a departure from the current practice, and so should be signaled by some expressed desire for strictness, rather than being the default.

Since no one can possibly modify the shared readtable binding from a loaded or compiled file, it is by construction compatible with current practice to force this binding at the beginning of the build. The big win is that it becomes safe to bind the *readtable* at the REPL, when it is today completely unsafe.

The only incompatible case is when you'd modify the *readtable* to something incompatible, e.g. that has a CL-in-CL implementation, then point the output translations to a different place, then run ASDF again in this crazy setup and expect it to work. To achieve the same effect, you'll now have to also bind *shared-readtable* to your magic CL-in-CL readtable. I don't believe anyone is doing this thing right now, and if they are, having to bind *shared-readtable* will be a small additional cost.

Note that I'm not arguing that syntax hygiene is a Bad Thing, only that it's not vanilla CL, so should not be the behavior you get from vanilla CL. By analogy, hygienic macros are arguably a Good Thing, but we keep CL macros unhygienic and make you GENSYM your way to hygiene if you want it. I could imagine a library that exports a hygienic variant of DEFMACRO, but I couldn't imagine making CL:DEFMACRO hygienic by default.

The thing with DEFMACRO is that it only affects people who use it. A bad *READTABLE* also pollutes people who don't use it. Binding *readtable* to *shared-readtable* saves users who bind it at the REPL, while being 100% backward compatible with those who modify the readtable structure. I think this should be the default, because it has great positive impact on those who have a non-standard *readtable* at the REPL, while having no imaginable impact on anyone else.

My patch also makes it trivial to make the readtable read-only instead, or a private copy for each system, or whatever you want. But I agree none of these can be the default, because of backward compatibility.

And whereas I'd like to bind syntax variables beside the *readtable*, I accept that this is not going to happen now or in the forseeable future, but I also make it trivial to implement, for those who want to try.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org As the Chinese say, 1001 words is worth more than a picture. — John McCarthy

(labels(({(] &rest [)(apply([ ])[))([(])(elt(]())]))(](<)( do-external-symbols(]`:cl)(push`,]`,<))(sort <`string<`:key`string))(}(} {)({`688({`875({`398()"~{~A ~}~%"(]()))}(+`,{(*)})))({`381)({`816`2/9))) ({`561()#'}`(874,948 7,6009 4862,370 10,12249)`(3,2 4,4 2,1 1,0)))

WTF!!

:D