It used to be possible to run multiprocessing servers like a webserver which coexisted with slime. This is no longer possible With current CVS slime, swank's WAIT-FOR-EVENT blocks all other processes from handling FD events
There is a simple test case, but here is a more realistic one: I'm attaching a program "t2.lisp" which seeks to illustrates the problem. This is a simple program which listens on a port [8128 by default] and prints a message to clients which connect to it.
-- In a bare lisp
* (load "t2.lisp")
LISTEN-FOR-CONNECTIONS creates a process which listens at port 8128.
-- run `telnet localhost 8128' from a shell, a few times.
This will result in the bogus service printing a message to the telnet client, as expected.
-- load slime in the lisp, and listen at port 4005
* (load "/path/to/swank-loader.lisp") * (swank-loader::init) * (swank::setup-server 4005 #'print nil nil (swank::find-external-format "latin-1-unix"))
-- Connect to the lisp from emacs using M-x slime-connect localhost 4005
-- run `telnet localhost 8128' from a shell.
This hangs until the slime connection exits.
This is a serious problem, and exposes many issues with SLIME's design. -- Madhu
* Madhu [2008-10-30 18:48+0100] writes:
It used to be possible to run multiprocessing servers like a webserver which coexisted with slime. This is no longer possible With current CVS slime, swank's WAIT-FOR-EVENT blocks all other processes from handling FD events
SLIME's WAIT-FOR-STREAMS calls SYS:SERVE-EVENT with a timeout of 0.2 seconds. That's shorter than LISP::*MAX-EVENT-TO-SEC*--which is 1 second--and consequently the LISP::*PERIODIC-POLLING-FUNCTION* isn't called.
There is a simple test case, but here is a more realistic one: I'm attaching a program "t2.lisp" which seeks to illustrates the problem. This is a simple program which listens on a port [8128 by default] and prints a message to clients which connect to it.
-- In a bare lisp
- (load "t2.lisp")
LISTEN-FOR-CONNECTIONS creates a process which listens at port 8128.
-- run `telnet localhost 8128' from a shell, a few times.
This will result in the bogus service printing a message to the telnet client, as expected.
-- load slime in the lisp, and listen at port 4005
- (load "/path/to/swank-loader.lisp")
- (swank-loader::init)
- (swank::setup-server 4005 #'print nil nil (swank::find-external-format "latin-1-unix"))
nil as *communication-style* is not the default.
-- Connect to the lisp from emacs using M-x slime-connect localhost 4005
-- run `telnet localhost 8128' from a shell.
This hangs until the slime connection exits.
It doesn't hang in the default configuration.
More precisely, it doesn't hang if I run SLIME with (eq *communication-style* :sigio). It hangs, with the default config, if the SLIME debugger is active.
It doesn't hang, even with active debugger, if (eq *communication-style* :spawn).
It doesn't hang, even with active debugger, if *max-event-to-sec* is 0 and *max-event-to-usec* is 1000.
It doesn't hang, even with active debugger, with (eq *communication-style* :sigio) if multiprocessing is initialized properly and the idle process runs. E.g. if you start the swank server with:
(swank::initialize-multiprocessing (lambda () (swank:create-server)))
This is a serious problem, and exposes many issues with SLIME's design.
Which issues?
Helmut.
* Helmut Eller m2d4hhn9lj.fsf@common-lisp.net : Wrote on Thu, 30 Oct 2008 20:53:12 +0100:
| SLIME's WAIT-FOR-STREAMS calls SYS:SERVE-EVENT with a timeout of 0.2 | seconds. That's shorter than LISP::*MAX-EVENT-TO-SEC*--which is 1 | second--and consequently the LISP::*PERIODIC-POLLING-FUNCTION* isn't | called.
This is the default configuration
|> This hangs until the slime connection exits. | | It doesn't hang in the default configuration.
The scenario I outlined above WAS the default configuration. Repeating the test without a .swank.lisp and calling CMUCL with lisp -noinit
* SWANK::*COMMUNICATION-STYLE* => :SIGIO
| More precisely, it doesn't hang if I run SLIME with | (eq *communication-style* :sigio). | It hangs, with the default config, if the SLIME debugger is active.
The scenrario I outlined abiove which illustrates the hanging did not have a debugger hanging.
| It doesn't hang, even with active debugger, if | (eq *communication-style* :spawn).
COMMUNICATION STYLE of SPAWN and SIGIO interfere in various non-trivial ways with the behaviour of the application especially if it is a multiprocessing webserver.
| It doesn't hang, even with active debugger, if *max-event-to-sec* is 0 | and *max-event-to-usec* is 1000.
I am sticking to the default configuration for now, to emphasise there is a problem, and there are issues.
| It doesn't hang, even with active debugger, with | (eq *communication-style* :sigio) if multiprocessing is | initialized properly and the idle process runs. E.g. if you start the | swank server with: | | (swank::initialize-multiprocessing (lambda () (swank:create-server)))
No, this is a hack in your personal setup. This precludes various other ways in which CMUCL can be used, introduces some problems in other scenarios.
You cannot impose your personal configuration on every application. The point I'm making here is slime used to work with a range of application configurations.
|> This is a serious problem, and exposes many issues with SLIME's design. | | Which issues?
I'm having a hard time getting you to even admit there is a problem. I suspect you are either extremely short sighted or are blind to defects.
-- Madhu
* Madhu [2008-10-31 02:59+0100] writes:
|> This hangs until the slime connection exits. | | It doesn't hang in the default configuration.
The scenario I outlined above WAS the default configuration. Repeating the test without a .swank.lisp and calling CMUCL with lisp -noinit
- SWANK::*COMMUNICATION-STYLE* => :SIGIO
If you start swank with swank::setup-server instead of swank:create-server your are using a non-default configuration. Your code passes nil as the communication-style argument to swank::setup-server.
| More precisely, it doesn't hang if I run SLIME with | (eq *communication-style* :sigio). | It hangs, with the default config, if the SLIME debugger is active.
The scenrario I outlined abiove which illustrates the hanging did not have a debugger hanging.
That scenario is using the "read and process requests sequentially" communication style.
| It doesn't hang, even with active debugger, if | (eq *communication-style* :spawn).
COMMUNICATION STYLE of SPAWN and SIGIO interfere in various non-trivial ways with the behaviour of the application especially if it is a multiprocessing webserver.
How does SPAWN interfere with a multiprocessing webserver? If you want to use multiprocessing, it would seem natural to use that style.
| It doesn't hang, even with active debugger, with | (eq *communication-style* :sigio) if multiprocessing is | initialized properly and the idle process runs. E.g. if you start the | swank server with: | | (swank::initialize-multiprocessing (lambda () (swank:create-server)))
No, this is a hack in your personal setup. This precludes various other ways in which CMUCL can be used, introduces some problems in other scenarios.
You cannot impose your personal configuration on every application. The point I'm making here is slime used to work with a range of application configurations.
If you want to use CMUCL's multiprocessing you should start the idle process somehow.
|> This is a serious problem, and exposes many issues with SLIME's design. | | Which issues?
I'm having a hard time getting you to even admit there is a problem. I suspect you are either extremely short sighted or are blind to defects.
Is it to much to ask to spell out "the many issues with SLIME's design"?
You claimed that "[SLIME] blocks all other processes from handling FD events". This is just not true, because SLIME calls SYS:SERVE-EVENTS. That CMUCL's SERVE-EVENTS only calls the thread scheduler after 1 second idleness isn't my fault. Or are you saying that SLIME shouldn't call SERVE-EVENTS?
Helmut.
* Helmut Eller m24p2tqh6l.fsf@common-lisp.net : Wrote on Fri, 31 Oct 2008 09:53:22 +0100:
| If you start swank with swank::setup-server instead of | swank:create-server your are using a non-default configuration. Your | code passes nil as the communication-style argument to | swank::setup-server.
You are right, I was incorrect in claiming the outlined scenario used the default sigio communication style.
|> | More precisely, it doesn't hang if I run SLIME with |> | (eq *communication-style* :sigio). |> | It hangs, with the default config, if the SLIME debugger is active. |> |> The scenrario I outlined abiove which illustrates the hanging did not |> have a debugger hanging. | | That scenario is using the "read and process requests sequentially" | communication style.
Right.
|> | It doesn't hang, even with active debugger, if |> | (eq *communication-style* :spawn). |> |> COMMUNICATION STYLE of SPAWN and SIGIO interfere in various non-trivial |> ways with the behaviour of the application especially if it is a |> multiprocessing webserver. | | How does SPAWN interfere with a multiprocessing webserver? If you | want to use multiprocessing, it would seem natural to use that style.
I have not used SPAWN with CMUCL MP recently. The sequential processing of slime requests (having only one outstanding slime request at a time) proved to be most robust (from the point of view of the application).
|> | (swank::initialize-multiprocessing (lambda () (swank:create-server))) |> |> No, this is a hack in your personal setup. This precludes various other |> ways in which CMUCL can be used, introduces some problems in other |> scenarios. |> |> You cannot impose your personal configuration on every application. The |> point I'm making here is slime used to work with a range of application |> configurations. | | If you want to use CMUCL's multiprocessing you should start the idle | process somehow.
There are outstanding "bugs" in stock CMUCL which make this unreliable and hose the MP system unless started at the REPL by the Initial process. No, it is not clear to me why there are bugs, and how to go about fixing them. In some other cases the "determinism" gains from not starting an idle process is desirable.
| Is it to much to ask to spell out "the many issues with SLIME's design"? | You claimed that "[SLIME] blocks all other processes from handling FD | events". This is just not true, because SLIME calls SYS:SERVE-EVENTS. | That CMUCL's SERVE-EVENTS only calls the thread scheduler after 1 second | idleness isn't my fault. Or are you saying that SLIME shouldn't call | SERVE-EVENTS?
No, on all three points. However this is less serious a problem than I inititally imagined. Thanks for the suggestions -- Madhu
-
Helmut Eller -
Madhu