[slime-devel] Asynchronous debugging
More on yesterday's experiments. The attached patch (1) adds :debug as a valid state transition from the idle state (2) adds SLIME-DEBUGGER-FUNCTION. This expects to be run in an environment in which the slime streams are available (e.g. from *slime-repl* or, I think, with C-c :) and returns a function that can be used for *debugger-hook* or *invoke-debugger-hook* Note that (setf sb-debug::*invoke-debugger-hook* (slime-debugger-function)) doesn't actually work from *slime-repl* because the hook has already been bound so you're not setting the global value. Answers on a postcard. (3) Changes eval-string to use *invoke-debugger-hook* instead of ordinary *debugger-hook* - because it's safe for BREAK, and also so it doesn't conflict with my existing (notionally portable) *debugger-hook* use in Araneida If I culd figure out an appropriate place to put it, I'd want to make this a (with-debugging-hooked ...) macro, but I think that needs build order frobbing The combined effect is that I can now use sldb when Araneida handlers cause errors. Setup recipe: CL-USER> (asdf:operate 'asdf:load-op 'araneida) .... CL-USER> (load "/home/dan/src/telent/araneida/doc/example") T CL-USER> (araneida:start-listening my-araneida-example::*listener*) #<SB-IMPL::HANDLER INPUT on descriptor 5: #<FUNCTION "CLOSURE" {96F8645}>> CL-USER> (setf araneida::*break-on-handler-errors* t) T CL-USER> (defvar *foo* (swank::slime-debugger-function)) #<FUNCTION "CLOSURE" {96F9815}> ;;; meanwhile in *inferior-lisp* * (setf sb-debug::*invoke-debugger-hook* cl-user::*foo*) I haven't applied this; it's more intended for discussion at present (particularly point (2)). Comments? -dan -- http://web.metacircles.com/cirCLe_CD - Free Software Lisp/Linux distro
Daniel Barlow <dan@telent.net> writes:
More on yesterday's experiments. The attached patch
Where is it? :)
(1) adds :debug as a valid state transition from the idle state
Perhaps a different name for the event, e.g, :async-debug, would be better. There is a certain risk that this transition hides (unrelated) bugs, if we use the same name. We might also consider to make this an out-of-band event.
(2) adds SLIME-DEBUGGER-FUNCTION. This expects to be run in an environment in which the slime streams are available (e.g. from *slime-repl* or, I think, with C-c :) and returns a function that can be used for *debugger-hook* or *invoke-debugger-hook*
Note that (setf sb-debug::*invoke-debugger-hook* (slime-debugger-function)) doesn't actually work from *slime-repl* because the hook has already been bound so you're not setting the global value. Answers on a postcard.
I assume you would like the set *invoke-debugger-hook*, so that it is used in your asynchronous handlers. We could modify the swank-server and enter an infinite loop instead of using fd-handlers for request processing. This "slime-loop" (idle-loop? break-loop?) would be similar to the sldb-loop. Assignments to special variables would then be visible as long as you are in the dynamic extend of the slime-loop. Is it acceptable that all your handlers run in the dynamic extend of the this loop? Another idea is that we allow support multiple sessions with multiple tcp connections (each session a dedicated connection). A asynchronous handler could then initiate a new session.
(3) Changes eval-string to use *invoke-debugger-hook* instead of ordinary *debugger-hook* - because it's safe for BREAK, and also so it doesn't conflict with my existing (notionally portable) *debugger-hook* use in Araneida
If I culd figure out an appropriate place to put it, I'd want to make this a (with-debugging-hooked ...) macro, but I think that needs build order frobbing
Until now we worked around this problem and used functions like call-with-debugging-hooked. The function is then defined later. Perhaps we should split the files again, so that the build order would be like: swank-early, swank-sbcl-early, swank, swank-sbcl. The *-early files contain the macro definitions a non portable package frobbing. Helmut.
Helmut Eller <e9626484@stud3.tuwien.ac.at> writes:
More on yesterday's experiments. The attached patch
Where is it? :)
One day I'll learn how to drive a mail reader.
(1) adds :debug as a valid state transition from the idle state
Perhaps a different name for the event, e.g, :async-debug, would be better. There is a certain risk that this transition hides (unrelated) bugs, if we use the same name. We might also consider to make this an out-of-band event.
You're more familiar with the protocol-level stuff than I am; I defer to your experience here.
I assume you would like the set *invoke-debugger-hook*, so that it is used in your asynchronous handlers.
Right
We could modify the swank-server and enter an infinite loop instead of using fd-handlers for request processing. This "slime-loop" (idle-loop? break-loop?) would be similar to the sldb-loop. Assignments to special variables would then be visible as long as you are in the dynamic extend of the slime-loop. Is it acceptable that all your handlers run in the dynamic extend of the this loop?
I don't think that's going to work in the context I have in mind: a long-lived server to which SLIME is connected periodically for debugging or upgrades. In a multithreaded server, all my request-answering processes are in their own threads which were started outside of a SLIME connection (perhaps at boot time) and will probably last for longer than the current SLIME session. Perhaps for HTTP it wouldn't hurt too much to tear them down and create new ones in a SLIME context, but for some more stateful protocol it'd be more pain. A secodary but probably solvable issue is that as currently implemented the sldb-loop blocks all serve-event processing, so new requests aren't getting answered when the debugger is active. This is for the moment livable with (and saves us from implementing the lock on the debugger that's next on the hacking list) but if simply connecting slime to the process is henceforth going to freeze it, that's not so great.
Another idea is that we allow support multiple sessions with multiple tcp connections (each session a dedicated connection). A asynchronous handler could then initiate a new session.
Not so wild about multiple TCP connections. My ultimate goal is to be able to slime across a network, which may require SSH tunnelling or other interesting configurations,
make this a (with-debugging-hooked ...) macro, but I think that
Until now we worked around this problem and used functions like call-with-debugging-hooked. The function is then defined later. Perhaps we should split the files again, so that the build order would be like: swank-early, swank-sbcl-early, swank, swank-sbcl. The
Yeah, I saw that, but the lambdas on lambdas were starting to make me dizzy, so I went with the simple solution for the time being. I know that refactoring the build is being talked about anyway: perhaps this can be addressed in that context -dan -- http://web.metacircles.com/cirCLe_CD - Free Software Lisp/Linux distro
Daniel Barlow <dan@telent.net> writes:
I don't think that's going to work in the context I have in mind: a long-lived server to which SLIME is connected periodically for debugging or upgrades. In a multithreaded server, all my request-answering processes are in their own threads which were started outside of a SLIME connection (perhaps at boot time) and will probably last for longer than the current SLIME session. Perhaps for HTTP it wouldn't hurt too much to tear them down and create new ones in a SLIME context, but for some more stateful protocol it'd be more pain.
What setup are you with right now - is it multithreaded? I'm wondering if you're binding *invoke-debugger-hook* globally for all threads (with potential race conditions), or if you're starting with a simpler somehow. Also, In a single-threaded SERVE-EVENT-based system, it might be nice if SLIME goes into an infinite loop as Helmut says, but is calling SYS:SERVE-EVENT instead of READ-FROM-EMACS. i.e. the usual SERVE-EVENT loop is running and serving the REPL, webserver, etc, but all within SLIME's dynamic binding setup. When the SLIME socket closes, we throw back out. (Or is that what you meant, Helmut?) -Luke
Luke Gorrie <luke@bluetail.com> writes:
What setup are you with right now - is it multithreaded?
Araneida does both serve-event and multithreaded: it just depends on which HTTP-LISTENER subclass you instantiate.
I'm wondering if you're binding *invoke-debugger-hook* globally for all threads (with potential race conditions), or if you're starting with a simpler somehow.
The former (but not rebinding it, just setting it). Where do you see the race conditions? I can't see wanting to do this from more than one place at a time. -dan -- http://web.metacircles.com/cirCLe_CD - Free Software Lisp/Linux distro
Luke Gorrie <luke@bluetail.com> writes:
In a single-threaded SERVE-EVENT-based system, it might be nice if SLIME goes into an infinite loop as Helmut says, but is calling SYS:SERVE-EVENT instead of READ-FROM-EMACS. i.e. the usual SERVE-EVENT loop is running and serving the REPL, webserver, etc, but all within SLIME's dynamic binding setup. When the SLIME socket closes, we throw back out. (Or is that what you meant, Helmut?)
Yes, that's what I meant. I just want to note that CMUCL calls SERVE-EVENT automatically if the file-descriptor is non-blocking. Helmut.
Daniel Barlow <dan@telent.net> writes:
I don't think that's going to work in the context I have in mind: a long-lived server to which SLIME is connected periodically for debugging or upgrades. In a multithreaded server, all my request-answering processes are in their own threads which were started outside of a SLIME connection (perhaps at boot time) and will probably last for longer than the current SLIME session. Perhaps for HTTP it wouldn't hurt too much to tear them down and create new ones in a SLIME context, but for some more stateful protocol it'd be more pain.
Thanks for the explanation. So, you'd like to set *invoke-debugger-hook* in one thread and make the change visible in the other threads, right? I guess then we need a set-invoke-debugger-hook slimefun, similar to set-package. Hmm... and probably a hook to detect when Emacs disconnects.
A secodary but probably solvable issue is that as currently implemented the sldb-loop blocks all serve-event processing, so new requests aren't getting answered when the debugger is active. This is for the moment livable with (and saves us from implementing the lock on the debugger that's next on the hacking list) but if simply connecting slime to the process is henceforth going to freeze it, that's not so great.
This should go a away if the file-descriptor is non-blocking.
Not so wild about multiple TCP connections. My ultimate goal is to be able to slime across a network, which may require SSH tunnelling or other interesting configurations,
Neither am I. But there are some advantages if we use multiple connections. E.g, we could use the same code for multi-threading support as for "multiple Lisp processes" support, locking would be easier, and we could disconnect if something goes wrong in one thread without disturbing the other threads. OTOH, Erlang seems to use a single connection for all threads. Maybe we should just steel whatever they do. Helmut.
participants (3)
-
Daniel Barlow -
Helmut Eller -
Luke Gorrie