Hi-diddly-ho,
Very nice cleanup and extension work Helmut :-)
I wonder - now that we have both input and output redirected to Emacs, would writing an `ielm'-style top-level just be a matter of some elisp hacking?
I've hacked the connection setup to run asynchronously, so M-x slime will try to connect in the background using a timer to retry. There's a new command `slime-disconnect' that will abort connection retrying or, if already connected, close the connection. This makes it easier to see what's going on and do manual fixes if e.g. the backend fails to compile or something.
Now I'm wondering about the next steps in the backend integration/factoring. There are two things I'd like to do:
Have a well-defined interface that the backend-specific code implements. Perhaps a `defcallback' macro that we use in swank.lisp where we can put a docstring and possibly some assertions about return values. We should also detect/warn for unimplemented backend functions. This seems pretty easy.
The other idea is not so clear, but I would like to factor all of the protocol-related code that sends and receives messages into common code. This is to make the protocol hackable without modifying/breaking all backends, and to get protocol invariants (like proper nesting of :debug and :debug-return) taken care of in one place.
I'm also now leaning towards Helmut's idea of using send/receive asynchronous message passing as the lowest layer of communication. Currently we have a hybrid where Emacs always talks to Lisp with RPC but Lisp can send state-specific asynchronous messages to Emacs. This is a bit funky in some cases, e.g. invoking a restart from Emacs would be better done as an asynchronous message than as an RPC since we don't want a result.
There's currently a bug in the protocol that you can see when trying to invoke restarts. In some cases the backends will want to send (:debug-condition <string>) messages to Emacs, but the Elisp code doesn't understand those messages anymore (due to my last "improvements"). In the automaton it would be nice to handle these messages in the `debugging' state, but since invoking a restart is done by RPC we end up in the `evaluating' state waiting for a result. So it seems nicer to control the debugger with asynchronous messages instead of RPCs.
So, um, that's pretty vague. Mostly written to publicly recant my previous suggestion that RPC is the way to go for everything :-)
One thing I did hacking-wise is to add more significant detection of protocol errors. If the Elisp code gets a message that it can't understand in its current state, it closes the connection and pops up a buffer with some debug info and a statement to the effect of "hey, you've found a bug!". Currently I'm getting this trying to invoke the `q'uit restart in the SBCL backend, due to the Elisp :debug-condition bug mentioned above (though I'm not sure why this is hitting a debug-condition in the first place - maybe a bug in the SBCL backend).
Cheers, Luke