Luke Gorrie luke@bluetail.com writes:

Hi-diddly-ho,

Very nice cleanup and extension work Helmut :-)

Thanks.

I wonder - now that we have both input and output redirected to Emacs, would writing an `ielm'-style top-level just be a matter of some elisp hacking?

Yes, mostly. Some things need a bit more thought, like what events should we accept in the read state, the proper semantics of FLUSH-INPUT, eofs etc. Also the stream implementation uses some very CMUCL specific buffer scheme (a fixed 512 byte vector) and it would probably good to make this a bit more flexible. But these issues are easier to resolve when we know how the REPL should look like. ielm seems to use comint, and I'm not sure if that's a good thing to do.

I've hacked the connection setup to run asynchronously, so M-x slime will try to connect in the background using a timer to retry. There's a new command `slime-disconnect' that will abort connection retrying or, if already connected, close the connection. This makes it easier to see what's going on and do manual fixes if e.g. the backend fails to compile or something.

This works pretty good now.

Now I'm wondering about the next steps in the backend integration/factoring. There are two things I'd like to do:

Have a well-defined interface that the backend-specific code implements. Perhaps a `defcallback' macro that we use in swank.lisp where we can put a docstring and possibly some assertions about return values. We should also detect/warn for unimplemented backend functions. This seems pretty easy.

Sounds good. Perhaps we could also add something like a default implementation for the functionality, and the backend can override this if the implementation has some special needs. Would be useful for things like read-next-form with a special case for CMUCL.

The other idea is not so clear, but I would like to factor all of the protocol-related code that sends and receives messages into common code. This is to make the protocol hackable without modifying/breaking all backends, and to get protocol invariants (like proper nesting of :debug and :debug-return) taken care of in one place.

Probably a good idea. sldb-loop seems to be the most complicated candidate for this refactoring, but doesn't look too hard.

I'm also now leaning towards Helmut's idea of using send/receive asynchronous message passing as the lowest layer of communication. Currently we have a hybrid where Emacs always talks to Lisp with RPC but Lisp can send state-specific asynchronous messages to Emacs. This is a bit funky in some cases, e.g. invoking a restart from Emacs would be better done as an asynchronous message than as an RPC since we don't want a result.

There's currently a bug in the protocol that you can see when trying to invoke restarts. In some cases the backends will want to send (:debug-condition <string>) messages to Emacs, but the Elisp code doesn't understand those messages anymore (due to my last "improvements"). In the automaton it would be nice to handle these messages in the `debugging' state, but since invoking a restart is done by RPC we end up in the `evaluating' state waiting for a result. So it seems nicer to control the debugger with asynchronous messages instead of RPCs.

Hehe, tricky. We could use asynchronous messages to call functions that are not supposed to return, right? If such a function enters the debugger, things would just work like they do now, but we would have to bind the *debugger-hook* somewhere. Alternatively we could add an event to the sate-machine that sends a RPC, but doesn't switch the state and ignores the :ok result.

Another thought: would it be possible to run the state machine from a an idle timer (with a very small timeout) instead from the process filter? Would that solve the problem with loosing input when someone throws out from the state machine?

So, um, that's pretty vague. Mostly written to publicly recant my previous suggestion that RPC is the way to go for everything :-)

One thing I did hacking-wise is to add more significant detection of protocol errors. If the Elisp code gets a message that it can't understand in its current state, it closes the connection and pops up a buffer with some debug info and a statement to the effect of "hey, you've found a bug!". Currently I'm getting this trying to invoke the `q'uit restart in the SBCL backend, due to the Elisp :debug-condition bug mentioned above (though I'm not sure why this is hitting a debug-condition in the first place - maybe a bug in the SBCL backend).

That's cool. We should perhaps ask before disconnecting, just in case someone has significant state on the stack.

--helmut.