Oh, and as for binding variables around files, this has subtly different meanings when compiling a file, loading its fasl or loading its source. And if done wrong it can easily break fasl concatenation or linking on ecl and mkcl, where you just can't bind around the loading of individual fasls, whereas with source loading you can't not bind, so now the user is responsible to make sure his code works both ways and the binding only really matters at compile-time—which kind of is already a development constraint, just unarticulated, unenforced, and undetected until a subtle bug hits someone.

In the end you can only protect the developer from himself so much, when you can't change the semantics of CL itself across twenty implementations many unmaintained.

On Thu, Feb 22, 2024, 11:16 Faré <fahree@gmail.com> wrote:

In the past, to introduce changes a fraction as compatibility-breaking (e.g. making utf-8 the default, or changing the internals of OPERATION), I would write up an explanation why the change is desired that anyone can read, discuss on the mailing list, write the code but keep it unmerged or disabled, make an announcement for the intended future breaking change, go over all of Quicklisp with Anton Vodonosov's cl-test-grid, locate all the offenders there, and send patches to each and everyone, wait a year if the old behavior used to be officially documented and supported, only a few weeks if it was a forbidden internals dependency, run the test again, give a last warning to those who didn't merge the patches, sometimes fork the systems left unmaintained to a new home, then finally make the change and announce it. And be ready to handle all the angry authors of proprietary code not in Quicklisp that you failed to previously contact.

For syntax-control (see my branch, that needs much love merging with 3.3.7), we never reached that point, if only because we were never satisfied with the complexity of the code (speaking for myself at least, I suspect for RPG also to a point, but he can opine otherwise if needed). There are many special variables beside *readtable* that control syntax (e.g. some systems try to make double-float the default—with "interesting" side effects to other systems compiled later, more subtle than the big obvious breakage when the readtable is wrong or corrupted). And users define more such variables as they extend the language. So now you want to maintain a dynamic list of symbols, some in packages not yet defined, that you want to track, save and restore. Next your users will want to bind the values of some of these variables around some systems, modules or components. The code is big and messy, and it isn't even obviously "the right thing" overall, though a lot of elements of it are, and every piece is needed somehow.

There's also the question of whether the default readtable should be THE "standard" readtable (immutable on some implementations, not on others), or some shared mutable readtable copied from it (will break many CCL extensions), or from the magic initial readtable that is not standard and that some will mutate (the most backward compatible option, so probably the one ASDF should pick, though it's ugly, unless you want to wrestle some more with the community).

Should we merge the whole messy thing that solves the entire problem? Only the small and clean part that is obviously right but clearly insufficient? Each time we make such a change, we have to pay a lot to overcome a big incompatibility hurdle. If the solution isn't fully satisfactory, then we are setting ourselves up for yet another round (or several) of such costly change(s).

I'm out of the Common Lisp community (moved to Gerbil Scheme), but whoever wants to tackle this challenge for good better makes sure they (and the community!) are ready to pay the price for the transition, and multiple times so if they don't get it right on the first try. Oh, and once you take on the job, you'll be hated by some part of the community whether you subsequently make the change or don't.

Good luck!

On Thu, Feb 22, 2024, 10:15 Robert Goldman <rpgoldman@sift.info> wrote:

I think this is still true, but... we cannot be discussing ASDF 3.2.1 here. It was released almost 7 years ago, and for whatever reason Zach refuses to update. The current version is 3.3.7

Please get a more recent ASDF and try again. I believe that this behavior is still in place: Faré proposed the "syntax control" patch to fix this, but we decided we had no easy path to introducing it, because it would break other code (admittedly not of good style) that relies on the old behavior of having the readtable setting leak out of one file into another. See https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/86

Any non-backwards compatible modification to ASDF -- even to the extent of raising a deprecation warning -- has led to temper-tantrums in the CL community...

On 21 Feb 2024, at 22:14, Scott@sympoiesis.com wrote:

Hi all! I just ran into something surprising. This is with ASDF 3.2.1, packaged with Quicklisp. I am using Named-Readtables. I had '*readtable*' set to a nonstandard readtable, then did quickload of a system unrelated to the one that defines and uses that readtable. The compilation failed; after I did '(in-readtable :common-lisp)' and tried again, it succeeded.

A quick glance at the ASDF source code shows that it binds '*readtable*' to a standard readtable in some cases, such as to read a '.asd' file, but not in 'uiop/lisp-build/compile-file*', nor in 'asdf/lisp-action:perform-lisp-compilation'. Wouldn't it make sense to do that?

-- Scott