How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes.
...an interesting idea.
Years ago (1984), I had dinner with Nils Nilsson (AI pioneer) and he mentioned the idea that he wanted to build a system that once alive would never be powered down again - in his view a minor but necessary prerequisite of an AI system.
It will be interesting to see what people will say about how to do this.
Regards to the List
Jack Harper Secure Outcomes Inc
Sent from my iPhone
On Dec 16, 2010, at 12:25, Steve Morin steve.morin@gmail.com wrote:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes. _______________________________________________ pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
Jack Harper wrote:
...an interesting idea.
Years ago (1984), I had dinner with Nils Nilsson (AI pioneer) and he mentioned the idea that he wanted to build a system that once alive would never be powered down again - in his view a minor but necessary prerequisite of an AI system.
It will be interesting to see what people will say about how to do this.
This is so interesting that I would like to answer it. I apologize for the fact that none of this has anything to do with Lisp per se.
What if the machine breaks or there is a power failure?
There has been some great work, I believe done cooperatively by Stanford and U.C. Berkeley, about how to deal with this. Do a web search for "crash-only systems". What you have to do is be able to quickly recover from such failures, getting you back to the same state you were in, or at least that latest state that you decided to checkpoint. Tecnology developed for database management systems is very helpful here. Some of the more recent Linux file systems have this property, ususally implemented by what DBMS people call redo-logging.
Bear in mind, though, that one reason people "reboot" things is because they *want* to get rid of the state, rather than to recover it! But *which* state should be reset and which should not? In traditional systems, some state is persistent, which means stored in places like files or DBMS's (or servers, sometimes), and the rest is "transient" and gets reset on a reboot.
Doing this is very crude. It just arbitarily does things like setting global variables to NIL or whatever their initial value was. Why do you think *that* is a particulary good thing to do? Well, it has something to do with the application programmer's *reason* to decide what to store in a file/DBMS and what not.
In the crash-only system, the good side is that you can control this: you can just call a function that resets whatever state you want to not preserve.
The drawback is that you might not succeed in enumerating all the ones that you'd like to see reset. So you might want to do something like putting a declaration on those things that you'd like to see reset (at those times when you otherwise would have done a reboot).
At this point, you may notice that traditional and crash-only systems are not as different as they initially appeared.
I should think about this more and make it a blog post, but first run it by the crash-only guys...
-- Dan
There has been some great work, I believe done cooperatively by Stanford and U.C. Berkeley, about how to deal with this. Do a web search for "crash-only systems". What you have to do is be able to quickly recover from such failures, getting you back to the same state you were in, or at least that latest state that you decided to checkpoint.
forgot to mention that one of the features I like most of CLISP is that its EXT:SAVEINITMEM will save a snapshot or an image without quiting the current image. http://www.gnu.org/software/clisp/impnotes/image.html
So you can snapshot on regular timing or on a trigger in the application (and then maybe ask rsyslog (on linux) to take care of archiving the snapshots.)
Ala'a Mohammad
On Sat, Dec 18, 2010 at 4:33 AM, Ala'a Mohammad amalawi@gmail.com wrote:
There has been some great work, I believe done cooperatively by Stanford and U.C. Berkeley, about how to deal with this. Do a web search for "crash-only systems". What you have to do is be able to quickly recover from such failures, getting you back to the same state you were in, or at least that latest state that you decided to checkpoint.
forgot to mention that one of the features I like most of CLISP is that its EXT:SAVEINITMEM will save a snapshot or an image without quiting the current image. http://www.gnu.org/software/clisp/impnotes/image.html
With some care and on Unix/Linux, you can use fork(2) a child process to create a copy of the currently running Lisp image and than save an image in the child. I have used that technique with SBCL and CMUCL, both in order to write out a Lisp images and to write out snapshots of large in-memory data structures without pausing or terminating system operation.
In general, I always keep a clean dump of the persistent data structures that I want to keep in addition to any dumped Lisp images. I use Lisp images to optimize startup times, and explicit serialization to carry data over from one system release to another. Snapshotting the relevant data also makes it possible to switch Lisp implementations (which is something that I've actually done, with live data, from cmucl to sbcl to ccl).
-Hans
On Thu, 16 Dec 2010, Steve Morin wrote:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes.
I run a Lisp webapp from detachtty. It leaves a Unix socket around so you can talk to the fake tty later. I have done live-updates using this.
http://www.cliki.net/detachtty
Whether you pass functions as functions or symbols in various places can have an impact on your ability to do a live update (symbols being an indirect reference, are more conducive).
-David
On Thu, Dec 16, 2010 at 12:25 PM, Steve Morin steve.morin@gmail.com wrote:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes.
The best generic answer is probably the following: if you have started something at the REPL, you can interrupt the process (Ctrl-C or something like that) and bet a "break" prompt, from which you can do everything you can do from the regular prompt: load/compile files, define functions, fix bugs &c &c &c. Then you should be able to continue the process you just interrupted.
A less generic answer would be to start an extra thread which would be listening on a port and then you can connect to that port to get a lisp prompt without interrupting all the other processes currently running. Then you can do everything (load/compile files, define functions, fix bugs &c &c &c) and the lisp should be able to figure out which processes have to be stopped for which actions (e.g., a process using CLOS might have to be stopped while low-level MOP stuff is redefined). You (and the lisp implementors) have to be careful, of course, about the safe points when things can be interrupted and watch out for the little things like if you interrupt lisp inside a recursive function F and redefine F, or if you redefine F by connecting to a separate thread, then you don't know whether the recursive calls to F will be using the new or the old definition of F unless you disassemble the old definition of F and find out whether the recursive call is compiled as a jump (old definition will be used) or as a (FUNCALL #'F) (also old definition) or as a (FUNCALL 'F) (new definition will be used).
Hi Steve,
when I am running server-style lisp applications, I actually do start swank (the lisp side of slime) in it and at any time I wish, I can connect with slime to it and interact.
Peter
The following blog post describes how to get Hunchentoot running on a Linux box with Swank running so that you can connect to the Lisp instance running the webserver through Slime:
http://blog.ponto-dot.com/2010/08/15/setting-up-common-lisp-on-a-web-server
On Thu, Dec 16, 2010 at 12:25 PM, Steve Morin steve.morin@gmail.com wrote:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes. _______________________________________________ pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes.
Why not just use slime/swank? You can open an ssh tunnel to your web server and use slime-connect.
hope this helps, Michael
On Fri, Dec 17, 2010 at 12:27 PM, Michael Bentley michael@stray-hound.comwrote:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes.
Why not just use slime/swank? You can open an ssh tunnel to your web server and use slime-connect.
Slime/swank is not very robust against version skew. This is especially a problem if you have multiple people who need to access the server, each of whom may be running a different version of slime. I assume people who are building swank into their servers are either running single-maintainer web sites, or are able to enforce uniformity of tools among all maintainers. And even then, you have to be willing to restart the server any time you update slime.
hope this helps, Michael
pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
This is an old issue! We now have quicklisp!
Cheers, Alexandre
On 17/12/2010, at 16:55, Gail Zacharias gz@clozure.com wrote:
Slime/swank is not very robust against version skew. This is especially a problem if you have multiple people who need to access the server, each of whom may be running a different version of slime. I assume people who are building swank into their servers are either running single-maintainer web sites, or are able to enforce uniformity of tools among all maintainers. And even then, you have to be willing to restart the server any time you update slime.
Alexandre Rademaker arademaker@gmail.com writes:
This is an old issue! We now have quicklisp!
Quicklisp doesn't solve that particular problem. You can still get different versions of slime between different people, even with Quicklisp. (It just depends who updates when.)
Zach
I mean, quicklisp made the update process easy enough to motivate developers to keep their slime up to date!
I am newbie and have to say that with quicklisp I was able to connect emacs with a remote lisp image in a couple of minutes. Just because it was easy to install slime in the server and client and to setup slime in the emacs on the client. Maybe for a pro quicklisp is not a big deal, but for me it is great!
Alexandre Sent from my iPhone
On 17/12/2010, at 18:18, Zach Beane xach@xach.com wrote:
Alexandre Rademaker arademaker@gmail.com writes:
This is an old issue! We now have quicklisp!
Quicklisp doesn't solve that particular problem. You can still get different versions of slime between different people, even with Quicklisp. (It just depends who updates when.)
Zach
On Fri, Dec 17, 2010 at 9:13 PM, Alexandre Rademaker arademaker@gmail.com wrote:
This is an old issue! We now have quicklisp!
How does Quicklisp solve the problem Gail described?
Steve Morin steve.morin@gmail.com writes:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes.
There are a lot of different ways. I use GNU screen http://www.gnu.org/software/screen/ to detach/reattach from interactive sessions running Emacs and slime attached to the Lisp webserver running http://wigflip.com/.
There's nothing magical about slime, though, or even really about the built-in repl of your implementation. Any way you can get data to the system can be used to create objects you can evaluate with EVAL.
For example, you could have a HTTP handler that accepts POST requests and takes a parameter as Lisp code to read with e.g. READ-FROM-STRING, evaluate with EVAL and return the result with e.g. PRINT-TO-STRING.
You wouldn't do that without thinking about the safety implications, of course...
Zach
On Fri, Dec 17, 2010 at 7:20 PM, Zach Beane xach@xach.com wrote:
There are a lot of different ways. I use GNU screen http://www.gnu.org/software/screen/ to detach/reattach from interactive sessions running Emacs and slime attached to the Lisp webserver running http://wigflip.com/.
A modern, well-maintained alternative to GNU screen is tmux (http://tmux.sourceforge.net/). I switched to tmux as GNU screen sometimes locks up sessions.
-Hans
On 12/17/2010 1:20 PM, Zach Beane wrote:
For example, you could have a HTTP handler that accepts POST requests and takes a parameter as Lisp code to read with e.g. READ-FROM-STRING, evaluate with EVAL and return the result with e.g. PRINT-TO-STRING.
We do something like this. For lisp websites my company makes, we have a password-protected admin section with some light UI to help us manage the site (turn logging levels up/down, clear caches, etc), and one of those tools is a "evaluate this code in the running lisp" textarea, with a dropdown to select what package it runs in. This is very rarely used to patch the site in emergency situations or for trivial changes where we don't want to bring down the site. This has bit us a few times, where we fixed a small bug directly in the running lisp and then forgot to publish the new code and had mystery regressions when the lisp process was restarted.
While having a long-running process on the server is appealing, we decided to spend our engineering time on making the lisp process easy to start, stop, restart, and deploy.
Personally, doing a lot of direct patching to a production system scares me. I worry about introducing bugs/hidden dependencies that wouldn't be represented in source code, and getting conflicts between multiple developers updating the same lisp.
Eventually the server is going to need a reboot or a have a hardware failure, and I want to have utmost confidence that my program will: 1) behave the same after a restart 2) behave the same when I deploy from source to another server
For development, we do run use swank over SSH tunnels, and run one copy of the website per developer.
Thanks,
Personally, doing a lot of direct patching to a production system scares me. I worry about introducing bugs/hidden dependencies that wouldn't be represented in source code, and getting conflicts between multiple developers updating the same lisp.
Maybe your button should only allow "load this asdf system". Then every code you use as a patch is a system that can be checked in -- hopefully just the same as you'd have in a fresh image, though sometime you need special magic (unintern, fmakunbound, etc.) to upgrade a live system that you don't need in a dead one.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Grâce à la bureaucratie et au socialisme, il n'y aura bientôt que deux partis en France: ceux qui vivent de l'impôt et ceux qui en meurent. [Because of bureaucracy and socialism, there will soon be only two parties left in France: those who live out of taxes, and those who die out of taxes.] — Achille Tournier, Pensées d'automne
Maybe your button should only allow "load this asdf system". Then every code you use as a patch is a system that can be checked in -- hopefully just the same as you'd have in a fresh image, though sometime you need special magic (unintern, fmakunbound, etc.) to upgrade a live system that you don't need in a dead one.
international-lisp-conference.org does precisely that (the "button" being a backdoor URL).
- nick
On Fri, 17 Dec 2010, Ryan Davis wrote:
While having a long-running process on the server is appealing, we decided to spend our engineering time on making the lisp process easy to start, stop, restart, and deploy.
I agree, very valuable. More so than live updating.
Personally, doing a lot of direct patching to a production system scares me. I worry about introducing bugs/hidden dependencies that wouldn't be represented in source code, and getting conflicts between multiple developers updating the same lisp.
One thing I do is rebuild the image from scratch, then test that (on a different port or system, maybe). If everything checks out, *and* I want to update without downtime (because it's only on 1 system for example), *then* I attach to the existing process and load up the new stuff.
If something goes wrong, well, I now have downtime anyway, but have the clean image available for a restart. But if everything appears fine, I might schedule a restart for some future time anyway.
So, there's a nice continuum to find a solution to fit your needs.
-David
On Fri, Dec 17, 2010 at 11:16 AM, Ryan Davis ryan@acceleration.net wrote:
We do something like this. For lisp websites my company makes, we have a password-protected admin section with some light UI to help us manage the site (turn logging levels up/down, clear caches, etc), and one of those tools is a "evaluate this code in the running lisp" textarea, with a dropdown to select what package it runs in. This is very rarely used to patch the site in emergency situations or for trivial changes where we don't want to bring down the site. This has bit us a few times, where we fixed a small bug directly in the running lisp and then forgot to publish the new code and had mystery regressions when the lisp process was restarted.
A really funny cautionary tale about this sort of thing:
http://thedailywtf.com/Articles/Designed-For-Reliability.aspx
-- Scott
Presumably this kind of thing is the reason for the Chaos Monkey:
http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-netflix-uses.php
-Peter
On Mon, Dec 20, 2010 at 6:37 PM, Scott L. Burson Scott@sympoiesis.com wrote:
On Fri, Dec 17, 2010 at 11:16 AM, Ryan Davis ryan@acceleration.net wrote:
We do something like this. For lisp websites my company makes, we have a password-protected admin section with some light UI to help us manage the site (turn logging levels up/down, clear caches, etc), and one of those tools is a "evaluate this code in the running lisp" textarea, with a dropdown to select what package it runs in. This is very rarely used to patch the site in emergency situations or for trivial changes where we don't want to bring down the site. This has bit us a few times, where we fixed a small bug directly in the running lisp and then forgot to publish the new code and had mystery regressions when the lisp process was restarted.
A really funny cautionary tale about this sort of thing:
http://thedailywtf.com/Articles/Designed-For-Reliability.aspx
-- Scott
pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
Yes, this kind of thing is essential if you want to design a system that stays up.
I have another fun thing along these lines. When you start a request that comes into the server, you assign it an integer. As the handling of the request reaches transaction boundaries (transactions on an underlying database management system), you decrease the counter by one and keep passing it along the chain of handling. (The chain might include sending messages from one component to another, and getting replies, and so on. For now I'm assuming that request handling only has one thread.)
Whenever the count reaches zero, the component that is currently handling the request is killed. The idea is to make sure that you test "all the places" where a crash might happen; that is, "all" with respect to the database state.
This tool is testing the system under a certain set of assumptions. It's assuming that the DBMS does what it's told to do. It's testing "stop" failures. It works best when the only side-effects are to the database system. If there are other side effects, e.g. the components get things into their cache that stay there and are used in subsequent requests, then you are less sure that you are testing out all possible paths.
I like to call these things "failure injection" (I didn't invent that term). We have other failure injection stuff all over our system. The "transaction ticking time bomb" one is just my favorite.
Using a random tool might not find all of these states. Random tools are great, but there are other useful test tools, too. In general, trying to "test all possible circumstances" is very, very hard; you never know what particular input values might be the ones that cause a problem, and then you have to worry about variable A having certain values while variable B has certain other values, leading to a combinatorial explosion of circumstances to test.
In my opinion, one of the best ways to deal with this problem is by having experienced Q/A people who have a knack for guessing what cases ought to be tested.
There's a lot I can say about this, but the main thing I'll say is that this is one of the reasons I have doubts about the wisdom of the methodology used at Facebook, in which there is no Q/A department, and programmers are expected to do their own Q/A. That's just one of the reasons.
It helps that at Facebook, you can roll out a new feature to a very small subset of users, and un-install it quickly if it's causing a problem, and usually if it doesn't work, that's not very important when there's only a very small number of early adopters. Things generally don't work that way with an airline reservation system. The methodologies that are suited for one situation are not necessarily those suitable for another.
-- Dan
Peter Seibel wrote:
Presumably this kind of thing is the reason for the Chaos Monkey:
http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-netflix-uses.php
-Peter
On Mon, Dec 20, 2010 at 6:37 PM, Scott L. Burson Scott@sympoiesis.com wrote:
On Fri, Dec 17, 2010 at 11:16 AM, Ryan Davis ryan@acceleration.net wrote:
We do something like this. For lisp websites my company makes, we have a password-protected admin section with some light UI to help us manage the site (turn logging levels up/down, clear caches, etc), and one of those tools is a "evaluate this code in the running lisp" textarea, with a dropdown to select what package it runs in. This is very rarely used to patch the site in emergency situations or for trivial changes where we don't want to bring down the site. This has bit us a few times, where we fixed a small bug directly in the running lisp and then forgot to publish the new code and had mystery regressions when the lisp process was restarted.
A really funny cautionary tale about this sort of thing:
http://thedailywtf.com/Articles/Designed-For-Reliability.aspx
-- Scott
pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
I just want to point to another possibility, however, it's specific to CMUCL using the remote package
http://common-lisp.net/project/cmucl/doc/cmu-user/ipc.html
Ala'a Mohammad
On Thu, Dec 16, 2010 at 9:25 PM, Steve Morin steve.morin@gmail.com wrote:
How to interact with a running lisp instance? I have been trying to figure this out. I know this is being done with slime. Does any one have any good pointer on this. I am thinking of writing a web application and would like to be able to update it on the fly for updates and bug fixes. _______________________________________________ pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
On Fri, Dec 17, 2010 at 8:01 PM, Ala'a Mohammad amalawi@gmail.com wrote:
I just want to point to another possibility, however, it's specific to CMUCL using the remote package
Now that you mention it, LispWorks has something similar, albeit a bit more simplistic: Just use (comm:start-up-server :service <some-port>) and there you are.
http://www.lispworks.com/documentation/lw60/LW/html/lw-578.htm
I usually use the detachtty approach, though.
Edi.
It depends on what you mean by "interact". Is it a program interacting or a human interacting. If you mean the latter, then Swank makes sense. If the former, Swank isn't needed and is probably overkill As Sam Steingold says:
A less generic answer would be to start an extra thread which would be listening on a port and then you can connect to that port to get a lisp prompt without interrupting all the other processes currently
So you'd only need a REPL talking to a stream.
Or you could send a form by HTTP. That can help with some problems. If you just have REPL on a stream, the boundary between forms is entirely ruled by the Lisp reader, and if there's any confusion between the client and the server about this (think "reader macros"!), then you could get out of sync and the whole thing stops working. HTTP gives you boundaries that you can re-sync to. There are nice portable Common Lisp package for both client (drakma) and server (hunchentoot).
In our own Common Lisp servers, we actually do both We have two things that we use for debugging or for emergency patching:
(1) Swank is pre-loaded into the server, so if we want to get in and look around, we can run Emacs with slime, and tell slime to connect over a TCP connection. As Gail points out, you have to be careful to use the version of Slime that corresponds to the version of Swank that was pre-loaded. We do not upgrade very often so that hasn't been a problem in practice but you definitely have to do that right.
(2) We have a simple way to send one HTTP request with a Lisp form in it, which is read and evaluated. (Zach suggested this.) This is ONLY for emergencies, i.e. it must be done quickly, e.g. because your system must be highly available.
You have to be very careful with this general eval mechanism. If there were time enough to test the fix, then we would not use this mechanism at all. We'd do the fix, commit it into the source control system, rebuild the code, and do a "hot upgrade", basically a rolling upgrade across the cluster of servers. There are version issues that you have to worry about here, and we have official guidelines about how to deal with that. Ryan Davis's reply about this went into detail and I agree with him 100%. David Owen's reply, I believe, was in respect to a setup where you aren't using clustering.
Ala'a Mohammad said:
I just want to point to another possibility, however, it's specific to CMUCL using the remote package
http://common-lisp.net/project/cmucl/doc/cmu-user/ipc.html
I haven't looked, but I would guess that using drakma and hunchentoot, it could be easy to make this portable between CL implementations.
-- Dan
On 17 December 2010 15:21, Daniel Weinreb dlw@itasoftware.com wrote:
(1) Swank is pre-loaded into the server, so if we want to get in and look around, we can run Emacs with slime, and tell slime to connect over a TCP connection. As Gail points out, you have to be careful to use the version of Slime that corresponds to the version of Swank that was pre-loaded. We do not upgrade very often so that hasn't been a problem in practice but you definitely have to do that right.
More importantly, we keep matching versions of swank and slime in our version control system. If you want to attach to a server, make sure you're using the appropriate SLIME. This way we could debug years-old production servers using obsolete versions of SWANK, if only we had any.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] As the Chinese say, 1001 words is worth more than a picture. — John McCarthy
Faré fahree@gmail.com writes:
More importantly, we keep matching versions of swank and slime in our version control system. If you want to attach to a server, make sure you're using the appropriate SLIME. This way we could debug years-old production servers using obsolete versions of SWANK, if only we had any.
Quicklisp does actually aim to help in this regard. Project archives and metadata are kept long-term, with the aim of going back to previous sets of working configurations as needed. That includes swank and slime. (Of course, there's no guarantee that the old software works with the new Lisp implementations, and Quicklisp currently doesn't make any effort to archive e.g. SBCL snapshots...)
The API to take advantage of this long-term stability hasn't been exposed, yet.
Zach
On Fri, Dec 17, 2010 at 2:35 PM, Faré fahree@gmail.com wrote:
More importantly, we keep matching versions of swank and slime in our version control system. If you want to attach to a server, make sure you're using the appropriate SLIME. This way we could debug years-old production servers using obsolete versions of SWANK, if only we had any.
We do the same thing.
This seems like the thread in which to ask whether anyone has found a straightforward way to switch Slime versions without restarting Emacs. Right now I usually have (under screen) one instance of Emacs for personal projects (for which I try to use the latest Slime and Swank) and one for work (where they do not get updated so frequently), and sometimes I need to start a third instance if I am doing work on an older maintenance branch of the software. It seems like there should be a way to switch between Slime versions within the same running Emacs, but I have yet to figure it out.
--Eli
On 17 dec 2010, at 22:15, Eli Naeher enaeher@gmail.com wrote:
Right now I usually have (under screen) one instance of Emacs for personal projects (for which I try to use the latest Slime and Swank) and one for work (where they do not get updated so frequently), and sometimes I need to start a third instance if I am doing work on an older maintenance branch of the software. It seems like there should be a way to switch between Slime versions within the same running Emacs, but I have yet to figure it out.
Not a direct question to Eli, but why is Slime so version specific anyway?
On Dec 17, 2010, at 4:57 PM, aerique@xs4all.nl wrote:
On 17 dec 2010, at 22:15, Eli Naeher enaeher@gmail.com wrote:
Right now I usually have (under screen) one instance of Emacs for personal projects (for which I try to use the latest Slime and Swank) and one for work (where they do not get updated so frequently), and sometimes I need to start a third instance if I am doing work on an older maintenance branch of the software. It seems like there should be a way to switch between Slime versions within the same running Emacs, but I have yet to figure it out.
Not a direct question to Eli, but why is Slime so version specific anyway?
Ya have to wonder if Slime/Swank should be used Protocol Buffers or Thrift to do their RPC. These both support versioned protocols, and it would be great to have a CL binding to Protocol Buffers.
On Sun, 19 Dec 2010, Scott McKay wrote:
On Dec 17, 2010, at 4:57 PM, aerique@xs4all.nl wrote:
On 17 dec 2010, at 22:15, Eli Naeher enaeher@gmail.com wrote:
Right now I usually have (under screen) one instance of Emacs for personal projects (for which I try to use the latest Slime and Swank) and one for work (where they do not get updated so frequently), and sometimes I need to start a third instance if I am doing work on an older maintenance branch of the software. It seems like there should be a way to switch between Slime versions within the same running Emacs, but I have yet to figure it out.
Not a direct question to Eli, but why is Slime so version specific anyway?
Ya have to wonder if Slime/Swank should be used Protocol Buffers or Thrift to do their RPC. These both support versioned protocols, and it would be great to have a CL binding to Protocol Buffers.
As I understand it, the wire protocol is generally the stable part of Slime. The RPC aspects (what the server commands do) is what has deep implications and changes frequently.
[aside] Protocol buffers are an unnecessarily complicated reinvention of the wheel. If the big G wasn't using them, everyone would be using one of the preexisting formats (e.g. XDR). See MapReduce for a related theme. IMO, a binding to Apache Camel or OpenSplice DDS would be much more interesting.
- Daniel
I ended up with a strong preference for using screen(1) and the standard REPL. In fact I have often thought about putting my employer's toy into a screen session even for the unattended server mode. The Lisp is difficult to shut up anyway and sometimes you have to play special tricks if there is no terminal on stdin for the REPL. With a screen session you can do much better post-mortem debugging, too. Stream synchronization from error messages also works as if there was a terminal (because there is a terminal). This can be handy when people do output not keeping in mind it might go into a buffered logfile or so.
SLIME is really too fragile for this.
Any kind of TCP listener/evaluator opens a huge can of worms about authorization. With screen(1) you will have to have a ssh login, so your ssh setup will take care of auth, with the same ssh agent.
The only pain is sending forms from programs but I think *that* is when you do a custom named pipe (or TCP) solution and you do not use the same thing for interacting with the Lisp and for non-interactive evaluations.
Martin
On Dec 19, 2010, at 10:46 AM, Daniel Herring wrote:
On Sun, 19 Dec 2010, Scott McKay wrote:
On Dec 17, 2010, at 4:57 PM, aerique@xs4all.nl wrote:
On 17 dec 2010, at 22:15, Eli Naeher enaeher@gmail.com wrote:
Right now I usually have (under screen) one instance of Emacs for personal projects (for which I try to use the latest Slime and Swank) and one for work (where they do not get updated so frequently), and sometimes I need to start a third instance if I am doing work on an older maintenance branch of the software. It seems like there should be a way to switch between Slime versions within the same running Emacs, but I have yet to figure it out.
Not a direct question to Eli, but why is Slime so version specific anyway?
Ya have to wonder if Slime/Swank should be used Protocol Buffers or Thrift to do their RPC. These both support versioned protocols, and it would be great to have a CL binding to Protocol Buffers.
As I understand it, the wire protocol is generally the stable part of Slime. The RPC aspects (what the server commands do) is what has deep implications and changes frequently.
But what I am saying is, use something that supports versioned wire protocols, and instead of willy-nilly changing the existing API frequently, change it in constrained ways, increment a version number, and use a new version of the wire protocol.
Yes, it's work.
[aside] Protocol buffers are an unnecessarily complicated reinvention of the wheel. If the big G wasn't using them, everyone would be using one of the preexisting formats (e.g. XDR). See MapReduce for a related theme. IMO, a binding to Apache Camel or OpenSplice DDS would be much more interesting.
- Daniel
Scott McKay wrote:
On Dec 19, 2010, at 10:46 AM, Daniel Herring wrote:
On Sun, 19 Dec 2010, Scott McKay wrote:
On Dec 17, 2010, at 4:57 PM, aerique@xs4all.nl wrote:
On 17 dec 2010, at 22:15, Eli Naeher enaeher@gmail.com wrote:
Right now I usually have (under screen) one instance of Emacs for personal projects (for which I try to use the latest Slime and Swank) and one for work (where they do not get updated so frequently), and sometimes I need to start a third instance if I am doing work on an older maintenance branch of the software. It seems like there should be a way to switch between Slime versions within the same running Emacs, but I have yet to figure it out.
Not a direct question to Eli, but why is Slime so version specific anyway?
Ya have to wonder if Slime/Swank should be used Protocol Buffers or Thrift to do their RPC. These both support versioned protocols, and it would be great to have a CL binding to Protocol Buffers.
As I understand it, the wire protocol is generally the stable part of Slime. The RPC aspects (what the server commands do) is what has deep implications and changes frequently.
But what I am saying is, use something that supports versioned wire protocols, and instead of willy-nilly changing the existing API frequently, change it in constrained ways, increment a version number, and use a new version of the wire protocol.
Yes, it's work.
Slime and Swank already check the versions when they start. It's part of the defined wire protocol. Well, actually, it's defined as part of the convention that's always used between Slime and Swank. So if you want to use Swank from a non-Slime client, you ought to put this in yourself.
Swank has a global variable named *swank-wire-protocol-version*. When Slime sends the connection-info command, Swank sends back a bunch of info including this value. Slime checks whether this equals slime-protocol-version, and if not, asks you "Protocol version mismatch. Continue anyway? " with a yes or no answer.
Both Slime and Swank get the protocol version by opening the file ChangeLog file and getting the first token in it, which is something like "2007-12-20".
The important issue is *when* they do it.
In Swank, it is done at the time that the swank-loader.lisp file is loaded. Under normal circumstances, i.e. when you start up a Common Lisp instance that Swank isn't already loaded into, it happens when you start up Slime, beacause that starts up Swank.
In Slime, there is a top-level form:
(setq slime-protocol-version (eval-when-compile (slime-changelog-date)))
I think this means at the time that swank.el is compiled into swank.elc.
The ChangeLog file is supposed to be updated with a new entry at the front every time a change is made to either Slime or Swank. So the code is very conservative about versions. You don't have to ask whether the change you just made would alter the protocol. As Daniel Herring pointed out, even if all the functions and arguments remain the same, their effect might differ from one version to the next in a way that matters to the client.
The downside to being so conservative is that you are forced to keep your Slime and Swank aligned any time you make a change. If you have Swank pre-loaded in your image, the client must use exactly the same version of Slime, or else subvert the checking mechanism (by altering the ChangeLog file) or something.
At least it looks that way to me.
-- Dan
[aside] Protocol buffers are an unnecessarily complicated reinvention of the wheel. If the big G wasn't using them, everyone would be using one of the preexisting formats (e.g. XDR). See MapReduce for a related theme. IMO, a binding to Apache Camel or OpenSplice DDS would be much more interesting.
(Although this is an interesting topic, I suggest that we not get into this on the pro-lisp list.)
- Daniel
pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
-
aerique＠xs4all.nl -
Ala'a Mohammad -
Alexandre Rademaker -
Daniel Herring -
Daniel Weinreb -
David Owen -
Edi Weitz -
Eli Naeher -
Faré -
Gail Zacharias -
Hans Hübner -
Jack Harper -
Martin Cracauer -
Michael Bentley -
Nick Levine -
Peter Herth -
Peter Seibel -
Ryan Davis -
Sam Steingold -
Scott L. Burson -
Scott McKay -
Steve Morin -
Tom Emerson -
Zach Beane