I realize the question in the subject line is not specific enough to elicit useful answers. I need to ask some questions about how swank uses streams, so that I can ensure that forthcoming changes to CCL coexist with swank in a civil fashion. Unfortunately, I don't know enough about swank to ask intelligent questions.
Basically, what's happening is that we're working on some changes to CCL that are intended to make inspection and debugging of running Cocoa apps more reliable and convenient. In the course of that work, some of the Clozure hackers have noticed inputs and outputs going to unexpected places when swank servers are running. They're complaining about streams getting snatched out from under them.
Doubtless this is because we don't properly understand what swank is doing with the streams. I frankly don't know how to frame an intelligent question on this subject, because I don't know enough about what swank is trying to accomplish and how it's going about it. So, what questions should I ask? My goal is to ensure that CCL controls the standard streams in a way it can rely on, while still allowing a swank server to be created and run.
If you guys would help me zero in on the right questions to ask you about swank's operation, I'd appreciate it.
-me
* mikel evins [2009-06-02 17:31+0200] writes:
Doubtless this is because we don't properly understand what swank is doing with the streams. I frankly don't know how to frame an intelligent question on this subject, because I don't know enough about what swank is trying to accomplish and how it's going about it.
Basically SWANK installs a custom stream in *STANDARD-OUTPUT* and that stream forwards any output to the slime-repl Emacs buffer. We usually do this with dynamic bindings like
(let ((*standard-output* <custom-stream>) (*standard-input* <custom-stream>) (*debug-io* <custom-stream>) (*query-io* <custom-stream>) (*terminal-io* <custom-stream>)) ...)
before evaluating code that (potentially) generates output. We actually use PROGV but that's a minor detail.
A more problematic feature is "global IO redirection". This installs our stream as the truly global value for *STANDARD-OUTPUT* etc. and also as the default value for new threads. The reason to do such a global thing at all is the desire to redirect the output of threads created by a HTTP server, or generally threads that are not created by us. Global IO redirection is not enabled by default.
So, what questions should I ask? My goal is to ensure that CCL controls the standard streams in a way it can rely on, while still allowing a swank server to be created and run.
With *globally-redirect-io* enabled you're probably out of luck because we take over control. As often in CL, this could be solved by some convention but there is no established convention.
If you guys would help me zero in on the right questions to ask you about swank's operation, I'd appreciate it.
Maybe you could think about the HTTP server example and where debug output of worker threads should go and how to control that.
Helmut.
On Jun 2, 2009, at 1:35 PM, Helmut Eller wrote:
- mikel evins [2009-06-02 17:31+0200] writes:
Doubtless this is because we don't properly understand what swank is doing with the streams. I frankly don't know how to frame an intelligent question on this subject, because I don't know enough about what swank is trying to accomplish and how it's going about it.
Basically SWANK installs a custom stream in *STANDARD-OUTPUT* and that stream forwards any output to the slime-repl Emacs buffer. We usually do this with dynamic bindings like
(let ((*standard-output* <custom-stream>) (*standard-input* <custom-stream>) (*debug-io* <custom-stream>) (*query-io* <custom-stream>) (*terminal-io* <custom-stream>)) ...)before evaluating code that (potentially) generates output. We actually use PROGV but that's a minor detail.
A more problematic feature is "global IO redirection". This installs our stream as the truly global value for *STANDARD-OUTPUT* etc. and also as the default value for new threads. The reason to do such a global thing at all is the desire to redirect the output of threads created by a HTTP server, or generally threads that are not created by us. Global IO redirection is not enabled by default.
So, what questions should I ask? My goal is to ensure that CCL controls the standard streams in a way it can rely on, while still allowing a swank server to be created and run.
With *globally-redirect-io* enabled you're probably out of luck because we take over control. As often in CL, this could be solved by some convention but there is no established convention.
If you guys would help me zero in on the right questions to ask you about swank's operation, I'd appreciate it.
Maybe you could think about the HTTP server example and where debug output of worker threads should go and how to control that.
The problem at present is that loading and running swank is a way to destabilize CCL's Cocoa IDE.
The Cocoa IDE relies on a socket connection to an external process for certain critical situations. For example, a bug in a Cocoa event- handler can cause a Cocoa app written in CCL to become unresponsive: no events are being processed, because the event-handling thread is blocked on a break loop. CCL solves this problem by presenting the break loop in a separate application process (presently a dumb glass TTY in a Cocoa window).
It appears that loading and running swank can cause this connection to become unreliable, apparently by shuffling streams underneath us. One solution, favored by some users, is "don't load swank." Some of us are SLIME users, though. We'd rather figure out how to make swank coexist peacefully with the rest of the Cocoa CCL environment. In fact, those of us who are SLIME users would prefer to further extend SLIME and swank, for example, to arrange for the event-thread error handlers to be able to pop open a buffer in which we can present the break-loop UI. However, speculating about any such feature extensions is premature until we can guarantee the integrity of lower-level required features that must work correctly whether or not swank and SLIME are present.
When the CCL Cocoa app starts up, it also starts its glass TTY process, AltConsole. It creates a socket pair, connecting AltConsole's standard output, standard input, and standard error to one end of the socket, and CCL's *TERMINAL-IO* to the other end. We require the ability to guarantee the integrity of those stream connections. Is there a way to guarantee that integrity when swank is loaded and a server created? I'm perfectly willing to change CCL code or swank code to accomplish it, if need be. One way or another, though, that integrity is a requirement at our end.
--me
* mikel evins [2009-06-04 20:40+0200] writes:
When the CCL Cocoa app starts up, it also starts its glass TTY process, AltConsole. It creates a socket pair, connecting AltConsole's standard output, standard input, and standard error to one end of the socket, and CCL's *TERMINAL-IO* to the other end. We require the ability to guarantee the integrity of those stream connections. Is there a way to guarantee that integrity when swank is loaded and a server created? I'm perfectly willing to change CCL code or swank code to accomplish it, if need be. One way or another, though, that integrity is a requirement at our end.
As long as you set swank:*globally-redirect-io* to nil (in ~/.swank.lisp or before calling swank-loader:init) it should be pretty save to load Swank. Setting swank:*global-debugger* to nil is probably also a good idea. Additionally you can make swank::with-io-redirection a noop to prevent Swank from dynamically rebinding stream variables. If all this doesn't help than the problem is probably not related to Swank's streams.
Helmut.
On Jun 4, 2009, at 3:37 PM, Helmut Eller wrote:
- mikel evins [2009-06-04 20:40+0200] writes:
When the CCL Cocoa app starts up, it also starts its glass TTY process, AltConsole. It creates a socket pair, connecting AltConsole's standard output, standard input, and standard error to one end of the socket, and CCL's *TERMINAL-IO* to the other end. We require the ability to guarantee the integrity of those stream connections. Is there a way to guarantee that integrity when swank is loaded and a server created? I'm perfectly willing to change CCL code or swank code to accomplish it, if need be. One way or another, though, that integrity is a requirement at our end.
As long as you set swank:*globally-redirect-io* to nil (in ~/.swank.lisp or before calling swank-loader:init) it should be pretty save to load Swank. Setting swank:*global-debugger* to nil is probably also a good idea. Additionally you can make swank::with-io-redirection a noop to prevent Swank from dynamically rebinding stream variables. If all this doesn't help than the problem is probably not related to Swank's streams.
Cool; thanks. After some further discussion, we're considering adopting a policy that makes use of AltConsole and Swank an either/or thing. In other words, if the user elects to load and start swank, we disable the AltConsole connection and assume that any non-Cocoa interaction with the Lisp will be through swank. We're hoping that if someone loads swank, they'll be comfortable using SLIME as the UI in cases where the Cocoa UI is wedged.
THanks for the respo0nses.
--me
-
Helmut Eller -
mikel evins