On Sun, Oct 20 2013, Alexander Popolitov wrote:

...

Hi everyone!

So, here is my proposed enhancement to swank (see patch-file in the attachment). The idea is, that executing arbitrary code on the lisp core, which swank allows by default, is not always desirable. Hence, patch does following things: 1) Add SWANK::*READER* variable, which, when bound to a function when calling SETUP-SERVER, causes this function to be used when reading forms in SWANK-RPC::READ-FORM 2) SWANK::DISPATCH-EVENT's DESTRUCTURING-CASE is substituted by more flexible lookup in the hashtable. If SWANK::*VALID-RPC* list is bound to new value when calling SETUP-SERVER, then in this particular server only those new RPC's will be active.

1st change is useful, I think, not only for securing the reader in a way that I did in my CL-SECURE-READ system, but also if you actually want learn SWANK talk some non-s-exp-based protocol.

So, what do you think? I think SLIME is a tool for developing and debugging Lisp code and because of that security is not our priority. E.g. I don't even have a ~/.slime-secret and I think that only few people bother to create one. Potentially improved security is not a sufficient argument for me.

I'm also not a fan of making SLIME super flexible. Especially if that flexibility is not actually used and regularly tested. Usually testing becomes more difficult with more flexibility. Actually, I have a suggestion, how to make testing easier. Now we have CL swank server and Emacs slime slient, which can talk to

On 10/21/2013 11:32 AM, Helmut Eller wrote: this server. What is missing, is CL client, which is able to talk to swank server. If it were present, then one could immediately write bunch of unit tests for all thinkable situations.

On the actual code: I don't think that we want to COMPILE our code at runtime. Using a hashtable instead of DESTRUCTURING-CASE might be reasonable, but it also has it's downsides, e.g. it needs a new global variable for the hashtable. True enough, I've rewritten this bit as a macro - now no explicit call to COMPILE is involved. There are also some situations, e.g. in the debugger, when some RPCs are processed selectively and others are queued. Making those situations table driven is probably difficult and possibly even harder to understand than what we have now. But how is this queing handled in DESTUCTURING-CASE approach? It seems to me, that given rpc gets executed, whenever its form matches a destructuring pattern, regardless of whether we are now in debugging context or not.

On the other hand, in table-driven approach one could introduce another table of actually "active" rpcs, in parallel to the table of merely "defined" ones. This way, binding this active-table to some restricted set of commands when in debugger-mode may produce desirable effect. I think, that what you actually have as SLIME + SWANK is not merely a tool for developing and debugging lisp code. It is a tool for developing and debugging lisp code, built on top of a framework to define network length-based s-exp-based rpc-based protocols. Strictily speaking, it is not yet so, but my patch seems to make it so. And the fact, that required fixes are so minimal, indicates that it is almost so. In this new version of patch I've also added customizability of a writer function (which was PRIN1-TO-STRING-FOR-EMACS). Thus, if one desires to roll his own protocol in CL, one can do the following: 1) write a reader-function, which accepts strings, and returns s-exp-s 2) write a writer-function, which accepts s-exps and returns strings 3) define table of rpcs and that's all, one gets a server which processes requests asynchronously with the rest lisp core.

Helmut

On Tue, Oct 22 2013, Alexander Popolitov wrote:

...

I'm also not a fan of making SLIME super flexible. Especially if that flexibility is not actually used and regularly tested. Usually testing becomes more difficult with more flexibility. Actually, I have a suggestion, how to make testing easier. Now we have CL swank server and Emacs slime slient, which can talk to

On 10/21/2013 11:32 AM, Helmut Eller wrote: this server. What is missing, is CL client, which is able to talk to swank server. If it were present, then one could immediately write bunch of unit tests for all thinkable situations.

True that might be useful. But we'd still have to write more tests with more flexibility.

...

There are also some situations, e.g. in the debugger, when some RPCs are processed selectively and others are queued. Making those situations table driven is probably difficult and possibly even harder to understand than what we have now. But how is this queing handled in DESTUCTURING-CASE approach? It seems to me, that given rpc gets executed, whenever its form matches a destructuring pattern, regardless of whether we are now in debugging context or not.

I whish I had documented this better and as I can hardly remember the details. Search WAIT-FOR-EVENT in the source. Those are situations that scan the queued events and (potentially) process events out of order. DESTRUCTURING-CASE is partly modelled after Erlang's selective receive, though it's less general and we have to do the "selective" part manually with RECEIVE-IF. We could probably create a new macro, say SELECTIVE-RECEIVE, that could then either use tables or CASE under the hood.

On the other hand, in table-driven approach one could introduce another table of actually "active" rpcs, in parallel to the table of merely "defined" ones. This way, binding this active-table to some restricted set of commands when in debugger-mode may produce desirable effect.

That would work but it requires a bunch of tables. I doubt that handling multiple tables and an essentially unknown set of events is particulary easy to understand. Maybe it would be clearer to define some generic functions (perhaps with progn method combination type) and call them at places that actually need more flexibility.

I think, that what you actually have as SLIME + SWANK is not merely a tool for developing and debugging lisp code. It is a tool for developing and debugging lisp code, built on top of a framework to define network length-based s-exp-based rpc-based protocols. Strictily speaking, it is not yet so, but my patch seems to make it so. And the fact, that required fixes are so minimal, indicates that it is almost so.

In this new version of patch I've also added customizability of a writer function (which was PRIN1-TO-STRING-FOR-EMACS). Thus, if one desires to roll his own protocol in CL, one can do the following: 1) write a reader-function, which accepts strings, and returns s-exp-s 2) write a writer-function, which accepts s-exps and returns strings 3) define table of rpcs and that's all, one gets a server which processes requests asynchronously with the rest lisp core.

We could do that, but the question remains: why should we do it? Is the "if one desires to roll his own protocol in CL" case actually important? As I said before, improved security or more flexibilty have drawbacks too. Helmut