I know Attila (among others) makes heavy use of custom readtables. I wonder how he currently deal with those? (I've cc-ed Attila. :-)

there's already a package->readtable mapping in swank.

we use a custom hu.dwim.common:in-package that also sets up the readtable (our asdf system class makes sure the default package is hu.dwim.common instead of cl-user or asdf-user when loading a file, gory details in hu.dwim.asdf).

we also have a (def package ...) that allows for specifying a readtable setup form for a package that is used by the custom in-package macro, and also updates the internal swank mapping when the hu.dwim.def+swank asdf system is loaded.

Hi!

Ah, good point. Budden, your current version completely sidesteps this mechanism. It should be easy to add it as a fallback though.

I looked to a code once again and did some testing. I used emacs code:

(defun show-guessed-readtable () (interactive) (message (slime-eval '(cl:string (editor-hints.named-readtables::readtable-name swank::*buffer-readtable*)))))

and now I'm almost sure that my code uses *readtable-alist* if it does not find (in-readtable) statement in a file. So fallback works already.

I also found a problem: I can't change readtable through (in-readtable) in a slime repl buffer. I guess *readtable* somehow gets bound around real evaluation, but I dont know how to avoid it. Currently I have no idea what to do. It looks like some special actions must be done for repl buffers as we must take *readtable*

So I created a pull request https://github.com/slime/slime/pull/259 (hope I did it right). But code needs fixing.

...

we also have a (def package ...) that allows for specifying a readtable setup form for a package that is used by the custom in-package macro, and also updates the internal swank mapping when the hu.dwim.def+swank asdf system is loaded.

I'm interested in this. I need something like this atm. Is it available somewhere?

it's in quicklisp, and if you prefer repos then it's also available at http://hub.darcs.net/hu.dwim

you need to load hu.dwim.def+swank.

if you just want to copy the machinery, then the key part is in integration/swank.lisp

hth,

On Wed, Jul 15, 2015 at 11:20 PM, 73budden . budden73@gmail.com wrote:

*readtable-alist* is just a kludge.

Agreed.

[...]

I suggested verbose, but correct approach to deal with multiple readtables vs source files. If my suggestion is accepted, it looks like *readtable-alist* must be removed from SWANK.

I think we should try and deprecate *readtable-alist* but keep it working in parallel with your new in-readtable mechanism. (The latter can take precedence when there is a conflict.)

Looking forward to your updated / cleaned up pull request!

Cheers,

Hi!

My opinion is that two approaches are incompatible. Problem is the same I have noted before: (in-package) form leads to implicit (and sometimes unpredictable) change of current readtable in SLIME. I think more discussion is required. First I thought I could suggest a mode switch which alters behaviour of (in-readtable), but it is implicit and this is bad. So second suggestion is the following: 1. Create new form, say (in-readtbl) which changes readtable, but does not affect *readtable-alist* 2. Create variable, *lock-readtable-alist*. If it is true, error is signalled on attempt to alter *readtable-alist* 3. Display readtable name on modeline.

My opinion is that two approaches are incompatible. Problem is the same I have noted before: (in-package) form leads to implicit (and sometimes unpredictable) change of current readtable in SLIME.

why would it be unpredictable? if there's a relevant entry in *readtable-alist*, then it changes the current readtable only in the dynamic extent of an operation (as opposed to e.g. globally, or in the dynamic extent of the REPL, or sumesuch).

i don't see the problem: compile-toplevel-form and friends look for an in-package form higher up in the buffer, does the usual dance which includes optionally rebinding whatever *readtable-alist* says as the current readtable. then the same is done for an in-readtable form, which may rebind another readtable and thus override *readtable-alist*. and then the whole thing is forgotten when the compilation is finished and the stack is unwound.

IOW, the two can live next to each other independently, in-readtable having priority over *readtable-alist*.

or am i missing something?

I think more discussion is required. First I thought I could suggest a mode switch which alters behaviour of (in-readtable), but it is implicit and this is bad. So second suggestion is the following:

1. Create new form, say (in-readtbl) which changes readtable, but does

not affect *readtable-alist* 2. Create variable, *lock-readtable-alist*. If it is true, error is signalled on attempt to alter *readtable-alist* 3. Display readtable name on modeline.

IMO, this is way too much unjustified complexity (or i'm missing something).

Problem is that when you issue (in-readtable) form, it can alter *readtable-alist*.

it certainly can, but does it? and if it does, then why? let's change it so that it doesn't alter *readtable-alist*!

E.g. let's assume that :foo package is assicitated to :foo-readtable. Then consider the following call sequence in REPL:

(in-package :foo) (in-readtable :bar-readtable)

Then :foo package is associated to bar-readtable.

why? that sounds confusing to me. i see no reason for in-readtable to touch *readtable-alist*.

Also, if there are two source files where package is the same while readtables are different between the files, readtable associtated to package changes implicitly as a side-effect of file compilation or loading.

that should not happen, i consider such a behavior a bug.

E.g. this can occur after we edit either file and then load asdf system which contains both files. Then consider we are going to SLIME REPL and do (in-package). Then we assume that readtable changed implicitly, but it can change to value from either file, in dependence of our editing activity.

*lock-readtable-alist* would protect from this situation.

that's unjustified complexity. plus a concurrency lock would not solve the problem you described above.

73budden,

I've re-implemented your latest idea here:

https://github.com/capitaomorte/sly-named-readtables

It works and the mode-line indicator idea is a nice touch (there's a screenshot).

It's probably still a bit sketchy though: the readtables found by in (in-readtable ...) forms take precedence of others found in *readtable-alist* (i.e. in-readtable binds *buffer-readtable* tighter).

I would like to understand more about this conflict, before I implement some protection like you suggested, whereby (in-readtable ...) bindings would error out if they discovered there was already another binding coming form *readtable-alist*.

I'm still skeptical as Attila is as to how this would lead to unpredictable or suprising behaviour. Is it only that SLIME-specific kludge in NAMED-READTABLES itself or something deeper?

Another thing I don't fully understand in your idea (but I did reimplement it that way) is why the readtable information has to go on every Emacs->Lisp request, i.e., why not only on commands that eval lisp and compile parts of buffers? Is compile-file affected as well, should it be?

João

On Sat, Aug 15, 2015 at 1:21 PM, 73budden . budden73@gmail.com wrote:

Hi!

It is SLIME-specific problem. I hope I have already described the problem in previous posts. E.g. you type in SLIME: ... Of course, we can not say it is completely unpredictible, but is rather hard to predict sometimes.

OK, I see, if does seem to be SLIME-specific so it shouldn't affect my implementation.

...

Another thing I don't fully understand in your idea (but I did reimplement it that way) is why the readtable information has to go on every Emacs->Lisp request,

E.g. request can contain (read-from-string), which can occur in proper readtable context. But frankly, I just copied package-related policy to readtables without any serious design work :)

Yes, but even if they contain READ-FROM-STRING or friends it should only need/use the new readtable if WITH-BUFFER-SYNTAX eventually gets used, which is a small subset of the Lisp-side operations. As you say, it's the same with the package setting though, so perhaps it does make sense to mimic it.

João Távora

It doesn't ignore it. If *READTABLE-ALIST* is indeed used for source-code navigation, than the CL code responsible for that is already enclosed in WITH-BUFFER-SYNTAX, and thus SLY will consider whatever readtable indication travelled down the wire.

BTW it's not SLY that provides support for NAMED-READTABLES, it's the external contrib I pointed to.

On Mon, Aug 17, 2015 at 11:51 PM, 73budden . budden73@gmail.com wrote:

I suppose that *readtable-alist* is used for proper navigation on SBCL sources. If so, *readtable-alist* is rather important. If SLY ignores *readtable-alist* this may lead to incorrect code navigation in SBCL sources. But I don't know exactly.

Solution I suggested is to add new "in-readtbl" form in addition to "in-readtable". And lock *readtable-alist* from changes somehow. I

it's been a long time i've seen such kludgery being suggested...

don't know how we implement locking *readtable-alist* without patching named-readtables. Patching named-readtables is out of our authority, but I believe we need to consider it (maybe apply a patch) if we want that the entire system would work well.

just make a fork of both repos, record the needed changes in them, and when it all works and is tested then start nagging the maintainers to include your patches.

there's no need for any kind of authority, just working code of a reasonable quality (note: i'm pretty sure anything like the above mentioned kludgery will be rejected).

Hi! I've forked named-readtables and removed the kludge which alters swank:*readtable-alist*, pull request is here:

https://github.com/melisgl/named-readtables/pull/8

and an appropriate pull request to SWANK:

https://github.com/slime/slime/pull/278

Start of the discussion can be found here:

http://thread.gmane.org/gmane.lisp.slime.devel/11574

With best rgards, Budden