- erlisp-devel@lists.common-lisp.net - mailman3.common-lisp.net

[erlisp-devel] SoC planning
by Eric Lavigne 31 Jul '05

31 Jul '05

This is a recent AOL instant messenger conversation with my Summer of Code mentor, Faré. Eric (06:55:52) FareTUNES: hi. How's progress? (06:56:26) smokydiamond: I'm still reading the source code. I didn't need to learn much about how Erlisp works to do compiler porting. (06:56:43) FareTUNES: ok (06:56:47) smokydiamond: It is clearly written, though, so I don't think this will be a problem. (06:56:50) FareTUNES: what's your next target? (06:56:56) smokydiamond: process linking (06:57:00) FareTUNES: great! (06:57:09) smokydiamond: followed by all the stuff that depends on that (06:57:57) FareTUNES: indeed (06:58:44) smokydiamond: I think this will be much more interesting than porting. (06:59:22) smokydiamond: Researching compiler features was challenging at times, but never much fun. (06:59:59) FareTUNES: indeed (07:00:02) FareTUNES: :) (07:00:20) FareTUNES: what about distributed thing? (07:00:36) FareTUNES: or forking (which is about the same) (07:01:01) smokydiamond: I don't have time to do both, and it sounds like everyone is more interested in process linking. (07:01:35) smokydiamond: Just 3 weeks left, and I still need to take care of documentation. (07:02:32) FareTUNES: yes, I agree that process linking is way more challenging (07:03:58) FareTUNES: actually, as soc advisor, I'd definitely choose process linking (07:04:52) smokydiamond: I actually made the decision mostly on your advice. Your choice was very clear from mailing list discussions. (07:05:26) smokydiamond: Dirk also thought process linking should come first. (07:05:39) FareTUNES: thanks (07:05:49) smokydiamond: And mailing list discussion picked up a lot when we discussed that topic. (07:05:57) FareTUNES: :) (07:06:38) smokydiamond: Well, I am happy to be doing it also. It really does sound like interesting work :) (07:07:47) FareTUNES: great! (07:08:05) FareTUNES: don't hesitate to contact the list and/or me for any trouble (07:08:17) FareTUNES: it may help you, and it may help us, too :) (07:08:34) smokydiamond: Thanks. Actually, I will be contacting the list soon whether or not I run into trouble. (07:08:56) smokydiamond: When I come up with a plan for doing this, I want to discuss it on the list first. (07:09:13) FareTUNES: ok (07:10:25) FareTUNES: great. gtg (07:10:29) FareTUNES: bye! (07:10:38) smokydiamond: Bye, Fare. (07:10:46) smokydiamond: I am going to post this conversation on the list. (07:11:01) FareTUNES logged out.

1 0

[erlisp-devel] process linking
by Eric Lavigne 28 Jul '05

28 Jul '05

There has been some good discussion over the last few days, and you've all given me a good idea of how Erlang deals with process linking and related error handling issues. Now I'm going to take a few days to study Erlisp's source code and consider how to proceed. I'll let you know if I have any questions. Thanks, Eric -- http://plaza.ufl.edu/lavigne/ http://www.lispnyc.org/summerofcode.html

1 0

Re: [erlisp-devel] Re: SoC status update please
by Faré 27 Jul '05

27 Jul '05

[Oops, had sent this reply only to Willem. That's the message he refers to previously] On 27/07/05, Willem Broekema <metawilm(a)gmail.com> wrote: > To check whether a _node_ is still running and connected, monitor_node > can be used; from > <http://www.erlang.org/download/erlang-book-part1.pdf> page 98: > > "If no connection exists, and monitor_node/2 is called, the system > will try to setup a connection and deliver a nodedown message if the > connection fails." > > To check whether a _process_ is still alive, linking to it and > checking for the corresponding EXIT message is the way to go. > OK. So the question is: what happens if you're linked to a process on another node, and that node fails? You get an EXIT message, don't you? Or do you just sit idly until timeout, and need to have another process explicitly monitor the remote node and kill you? I guess that in Lisp, these details are handled with a proper meta-level protocol. Is there such a protocol in Erlang? > By the way, my experience with Erlang is mostly in trying to develop a > Gnutella client a few years ago. To the reasonable nice original > Gnutella protocol, many ill-specified ad-hoc extensions were added by > the existing clients. Implementing these extensions, which is > necessary in order to play well with the other clients, became less > and less interesting, so I never finished it. These sounds like the general story of any and all computing projects: interfacing with the horde out there is boring. Which can be seen as an opportunity and a challenge: can we build socio-technical infrastructures which allow to develop code that isn't boring? At least, the company I'm at has found a way to alleviate the bore: make money. > Although I still find > Erlang interesting, these days I'm more interested in Common Lisp. Wait till you have Erlisp, then. Or better: contribute -- your knowledge, for instance: you seem like the person who knows where to quickly find answers to the questions about Erlang :-) [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Work harder! Millions on welfare depend on you.

1 0

[erlisp-devel] Re: SoC status update please
by Joel Reymont 27 Jul '05

27 Jul '05

According to the Erlang heavy-weights... Begin forwarded message: > From: "Ulf Wiger" > Date: July 27, 2005 3:34:13 PM GMT+02:00 > > Den 2005-07-27 11:10:53 skrev Joel Reymont <joelr1(a)gmail.com>: > > >> Uffe, >> >> Can you confirm the keep-alive bit? How does Erlang know that a >> node is dead? >> >> Thanks, Joel >> > > See erl -man kernel > > "net_ticktime = TickTime <optional> > > "Specifies the net_kernel tick time. TickTime is given in seconds. > Once every TickTime/4 second, all connected nodes are ticked (if > anything else has been written to a node) and if nothing has been > received from another node within the last four (4) tick times > that node is considered to be down. This ensures that nodes which > are not responding, for reasons such as hardware errors, are > considered to be down. > > The time T, in which a node that is not responding is detected, > is calculated as: MinT < T < MaxT where: > > > MinT = TickTime - TickTime / 4 > MaxT = TickTime + TickTime / 4" > > It should be "if nothing else has been written to the node". > The inet driver keeps counters of packets sent and received. > If no packets have been sent since the last sample, a tick > is sent. > > /Uffe > -- http://wagerlabs.com/uptick

1 0

Re: [erlisp-devel] Re: SoC status update please
by Willem Broekema 27 Jul '05

27 Jul '05

On 7/27/05, Faré <fahree(a)gmail.com> wrote: > OK. So the question is: what happens if you're linked to a process on > another node, and that node fails? You get an EXIT message, don't you? If the communication with a monitored node fails, but that node is still running its processes, then you get at least a {nodedown, <node>} message. I don't know if you get EXIT messages for the linked processes on that node: on the one hand, the process has not exited yet; on the other hand, if the process exists later on, the EXIT message can not be sent at that time -- so I could imagine either case. It's important to know that a process needs to explicitly state that it wants to receive EXIT signals from linked processes, using "process_flag(trap_exit, true)" in the process that should receive the EXIT signals. If a process P has not set this flag, an exiting linked process P2 will also cause the process P to exit (unless the exit reason of the exiting linked process is "normal"). By setting the flag, process P override this default cascading effect and instead of exiting P will get the EXIT message. > Or do you just sit idly until timeout, and need to have another > process explicitly monitor the remote node and kill you? I guess that > in Lisp, these details are handled with a proper meta-level protocol. > Is there such a protocol in Erlang? I think the behaviour of a single process regarding failures etc is not very variable, so instead of having a meta-protocol at the level of individual processes, there are protocols of how sets of processes behave in an application, in terms of "supervision trees". This deals with automatically restarting failed processes or subsets of supervised processes, and also a cascading restarting into higher application levels when an error persists. Individual processes that implement a "standard" entity like a server or finite-state-machine can save much code by being implemented using a "behaviour". A "supervisor" is another such "standard" entity. Other than the gen_server and gen_fsm behaviours, I have not used these pretty advanced features, so I don't know more about them than what is documented in the document below. "OTP Design Principles" <http://erlang.se/doc/doc-5.4.8/doc/design_principles/part_frame.html>. The information there can be quite overwhelming. I found the following tutorial very useful for understanding the value and usage of the "gen_server behaviour": http://www.duomark.com/erlang/tutorials/proxy.html - Willem

1 0

[erlisp-devel] Re: SoC status update please
by Eric Lavigne 27 Jul '05

27 Jul '05

> > CMUCL integration is complete. > Great. Do you have process kill working in CMUCL, SBCL, Allegro, in a > way that is safe wrt locks and that propagates along the process tree? > That's a major feature required of an erlang implementation. I have not done any work on process killing. As far as I know, it is not a part of ErLisp right now. This will definitely require a kill-thread function in compatibility.lisp. Beyond that I'm not sure how to procede. Manually calling kill-thread? Automatically calling it following a time-out? Also, I would like to give each process the responsibility for killing its child processes, but if it's being killed because it is no longer responsive then that won't work... I would like to see some discussion on this topic in erlisp-devel because I have no experience with such situations. > > This leads me to wonder how they do a reliable detection of a remote > node being dead, as opposed to the communication channel being down -- > and how they cope with a mistake between the two. Surely the point is > tackled somewhere in some Erlang documentation... Best I can think of is for the node itself to be represented as a process whose only job is communication with the outside world. Since the user won't control this process directly, it should be possible to make it fairly durable so that we can assume it is alive. *crosses fingers* > > > Next up is CLisp. > I don't think clisp has a complete threading implementation yet. Does > it? If it doesn't, then it might be time to begin a distributed > implementation -- and to fork clisp processes as a way to build new > threads. It looks like CLisp threading is very much a work in progress. I didn't research this well enough in advance. Forking CLisp processes is a nice idea, but we might be better off just letting it go until CLisp has better thread support. The original reason for including CLisp was so that Windows users would have access to ErLisp. Allegro support solves that problem well enough for now. So what next? Dirk, is ErLisp already have some sort of slot waiting for process-linking code? Or will we be designing something from scratch? I'm thinking that my next step is to read Faré's thesis again. A robust distributed process management system does not sound like an easy task. Eric -- http://plaza.ufl.edu/lavigne/ http://www.lispnyc.org/summerofcode.html

5 5

[erlisp-devel] Description of pushed patches.
by Dirk Gerrits 26 Jul '05

26 Jul '05

cmucl-threads Eric Lavigne <lavigne.eric(a)gmail.com>**20050723181545 Modified threading procedures in compatibility.lisp to support CMU Common Lisp.

4 7

[erlisp-devel] Re: patch for cmucl compatibility
by Eric Lavigne 25 Jul '05

25 Jul '05

> > This patch adds support for cmucl. All tests pass in cmucl. I have not > > retested under allegro or sbcl, but it should not be an issue since > > all changes are prefaced by #+cmu. > > All tests pass for me too, in both CMUCL 19a and SBCL 0.9.2. Your patch > looks good so I've pushed it. About 'CMUCL gates' though, I don't know. > Taking a quick look at code/multi-proc.lisp I'd say enable-/ > disable-process or process-add/revoke-run-reason might be related, but > I'm just not sure. I'd have to read it more carefully. If wait queues stored lists of waiting processes, then notify could involve adding run reasons to all waiting processes. Sounds complicated, but in terms of speed it's probably a lot better than letting those waiting threads remain active. > > Oh BTW, SBCL 0.9.3 will probably break Erlisp, as they've switched from > thread IDs to thread objects in the current CVS, as far as I've been > told. Should be a rather simple fix though, when the time comes. It doesn't sound too difficult. Where did you find information on future versions, though? > > So what's on your schedule now? Threads for more compilers? Process > linking? Distributed programming? Just let the mailing list know, and > I'll write up a short blog post. (That reminds me, I still need to > forward the soc-erlisp e-mails...) Threads for one more compiler (clisp), then distributed programming. What is process linking? Do you have any ideas about what the interface should look like for distributed programming? I've seen some confusing material on erlang websites which suggested that an erlang programmer could write a concurrent program without knowing whether it would be run on one computer (threads) or spread over several. Sounds nice but I never figured out how that worked. > > Oh BTW, could you check whether you can checkout the patch now? I still > have no idea what went wrong last time... > I tried pulling the latest patch. No problems. Eric -- http://plaza.ufl.edu/lavigne/ http://www.lispnyc.org/summerofcode.html

3 7

[erlisp-devel] Description of pushed patches.
by dirk＠dirkgerrits.com 13 Jul '05

13 Jul '05

Expanded timeout clause syntax to support time-units. Dirk Gerrits <dirk(a)dirkgerrits.com>**20050615235950] allegro-threads Eric Lavigne <lavigne.eric(a)gmail.com>**20050711110548 Modified threading procedures in compatibility.lisp to support Allegro Common Lisp. Added structure erlisp-event to emulate SBCL's wait-queue. Added low-level mutex operations (lock-mutex and unlock-mutex) to support erlisp-event.

3 9

[erlisp-devel] mistake in with-timeout?
by Eric Lavigne 11 Jul '05

11 Jul '05

Dear Erlisp developers, I need some help to figure out a problem with my code. I am trying to add Allegro support to Erlisp, but I ran into some trouble as shown in this test result: ...........f........................ Did 36 checks. Pass: 35 (97%) Skip: 0 (0%) Fail: 1 (2%) Failure Details: RECEIVE-WITH-TIMEOUT []: NIL. I suspect my mistake is in erlisp:with-timeout. Here is a copy of that macro: (defmacro with-timeout ((milliseconds &rest after-timeout) &body body) #+sbcl `(sb-ext:with-timeout (/ ,milliseconds 1000) (handler-case (progn ,@body) (sb-ext:timeout () ,@after-timeout))) #+allegro `(mp:with-timeout ((/ ,milliseconds 1000) ,@after-timeout) ,@body) #-(or sbcl allegro) (error "Timeouts are not yet implemented for your compiler.")) The part I added (probably broken) is prefaced by #+allegro. Dirk's working code is prefaced by #+sbcl. Documentation on the allegro with-timeout macro is available here: http://www.franz.com/support/documentation/6.2/doc/pages/operators/mp/with-… A complete copy of compatibility.lisp (the only modified file) is available here: http://plaza.ufl.edu/lavigne/compatibility.lisp Hopefully you can figure out my mistake so Erlisp can work on one more compiler. I would appreciate any ideas about what might be wrong. Eric -- plaza.ufl.edu/lavigne/

2 2