I haven't fully grokked the code changes in the syntax-control branch yet. However, I have read over the design notes in TODO, and pulled them out into a separate file, which I am attaching here. I have annotated with a number of open questions, and added a bullet point to document rationale, which is currently mostly a place-holder for a statement.
WRT my concerns about the syntax-control modifications:
I have done some rooting around and have found a few libraries that are intended to set up a system for interactive use with a special readtable. These are things like expert system shells, etc.
So at some point near the end of their system load, they will set *READTABLE* prior to finishing the load and dropping out for the user to deal with them.
I can see that someone might not like this -- indeed, I'd rather there was a (START-FOO-SHELL) command instead. On the other hand, note that these are not really LIBRARIES; these are SYSTEMS. These are not intended to be slurped into something else: the programmer expects that his/her system will load, set up things for an interactive user, and then hand back control to the user through the REPL.
Isn't the syntax control approach fundamentally incompatible with this?
I don't think this is an especially tasty thing to do, but I'm also not convinced that it's ASDF's job to break arbitrary CL anti-patterns. Changing this behavior is likely to give very confusing bugs that won't be obvious to the poor user.
On the other hand, I see files where there's something that sets the READTABLE obviously where what's intended is to simply set the readtable for the compiling and loading of this one file, and leaking that change is wrong.
Note that a programmer who wants to be clean about this, in the current world, can set the readtable at the top of a file, and restore it at the bottom. If we take away the current behavior, the programmer who wants to set things up for the user is just completely out of luck. Or, maybe not: the very sophisticated ASDF user could add a PERFORM method on LOAD-OP on the interactive system. I think. I haven't grokked what's going on in syntax-control well enough.
I am in a quandary about this. IMO the problem is a hole in the CL spec, which doesn't provide a good way to distinguish between two different intended uses for readtables.
Before merging the syntax-control mods, I need to be convinced that it's appropriate for the ASDF community to force a solution to this problem on the CL community, instead of leaving the CL community to pick its way through the issue as it has to date.