This patch adds support for cmucl. All tests pass in cmucl. I have not retested under allegro or sbcl, but it should not be an issue since all changes are prefaced by #+cmu.
All tests pass for me too, in both CMUCL 19a and SBCL 0.9.2. Your patch looks good so I've pushed it. About 'CMUCL gates' though, I don't know. Taking a quick look at code/multi-proc.lisp I'd say enable-/ disable-process or process-add/revoke-run-reason might be related, but I'm just not sure. I'd have to read it more carefully.
If wait queues stored lists of waiting processes, then notify could involve adding run reasons to all waiting processes. Sounds complicated, but in terms of speed it's probably a lot better than letting those waiting threads remain active.
Oh BTW, SBCL 0.9.3 will probably break Erlisp, as they've switched from thread IDs to thread objects in the current CVS, as far as I've been told. Should be a rather simple fix though, when the time comes.
It doesn't sound too difficult. Where did you find information on future versions, though?
So what's on your schedule now? Threads for more compilers? Process linking? Distributed programming? Just let the mailing list know, and I'll write up a short blog post. (That reminds me, I still need to forward the soc-erlisp e-mails...)
Threads for one more compiler (clisp), then distributed programming. What is process linking?
Do you have any ideas about what the interface should look like for distributed programming? I've seen some confusing material on erlang websites which suggested that an erlang programmer could write a concurrent program without knowing whether it would be run on one computer (threads) or spread over several. Sounds nice but I never figured out how that worked.
Oh BTW, could you check whether you can checkout the patch now? I still have no idea what went wrong last time...
I tried pulling the latest patch. No problems.
Eric
Eric Lavigne wrote:
Taking a quick look at code/multi-proc.lisp I'd say enable-/ disable-process or process-add/revoke-run-reason might be related, but I'm just not sure. I'd have to read it more carefully.
If wait queues stored lists of waiting processes, then notify could involve adding run reasons to all waiting processes. Sounds complicated, but in terms of speed it's probably a lot better than letting those waiting threads remain active.
The way Erlisp uses wait queues, only one process will ever be waiting on it (the one doing the RECEIVE), though many processes can make it wake up (the ones doing SEND). Does that help? Again, I haven't studied CMUCL's threading mechanisms.
Oh BTW, SBCL 0.9.3 will probably break Erlisp, as they've switched from thread IDs to thread objects in the current CVS, as far as I've been told. Should be a rather simple fix though, when the time comes.
It doesn't sound too difficult. Where did you find information on future versions, though?
Mailing list: http://lists.sourceforge.net/lists/listinfo/sbcl-devel
IRC: #lisp at irc.freenode.net
So what's on your schedule now? Threads for more compilers? Process linking? Distributed programming? Just let the mailing list know, and I'll write up a short blog post. (That reminds me, I still need to forward the soc-erlisp e-mails...)
Threads for one more compiler (clisp), then distributed programming.
Fine with me, though you'd then have to do process linking for both parallel /and/ distributed programming afterwards. You may want to work on 'parallel process linking' before working on the distributed programming stuff.
What is process linking?
Process linking is one of Erlang's most important features. Faré said he had discussed this with you. This is from the soc-erlisp e-mail http://parenpower.com/mailman/private/soc-erlisp/2005-July/000015.html: (BTW, why are these archives password protected?)
Dirk Gerrits wrote:
As for non-distributed Erlisp, "creating a new thread in the existing Lisp image" is already supported on SBCL 0.9.1, and shouldn't be too hard to port to other Lisps and upcoming SBCL releases. There is still at least one very big missing feature though, and that's linking of processes. (See http://dirkgerrits.com/programming/erlisp/roadmap/ under "Error handling".)
Faré wrote:
Linking of processes is a very important feature of the Erlang programming model. The difficulty being that of killing a thread when it's in the middle of some observable side-effect. This require proper locking and/or proper recovery mechanisms (roll back or roll forward). I've discussed the issue with Eric. Basically, we can either refrain from killing outside of safe-points (notably invoked at Erlisp function calls), or we can require processes to hold a lock while they are doing any kind of non-thread-local side-effect.
Do you have any ideas about what the interface should look like for distributed programming? I've seen some confusing material on erlang websites which suggested that an erlang programmer could write a concurrent program without knowing whether it would be run on one computer (threads) or spread over several. Sounds nice but I never figured out how that worked.
I suggest getting a hold of (borrowing / buying second hand) the book Concurrent Programming in Erlang, Second Edition.
Anyway, the way that works is that Erlang processes live on /nodes/. If you send a message to a process on the same node, what will happen is probably similar to what Erlisp does now. If you send a message to a process on a different node, the message will go over the network. The syntax is identical, so that's probably what those websites mean. And you're supposed to assume the semantics are identical too, that is, message passing can fail in either case. In practice I'd think that message passing /actually/ failing is very, very rare unless the receiver is on a different node though. So you still need to design for distributed programming, but when you do, your code can run both locally and distributedly.
If you look at src/messaging.lisp, you'll see that the mechanisms for this are already there. SEND calls SEND-USING-NODES to do the work, which has the following specializations:
* from a remote node Gives the error "Can only send messages from a local process."
* to a remote node Gives the error "Distribution is not implemented yet."
* to a threaded process on a local node Does the whole mutex/mailbox/wait-queue thing.
You'd want to replace the second one. ;)
- Dirk
Taking a quick look at code/multi-proc.lisp I'd say enable-/ disable-process or process-add/revoke-run-reason might be related, but I'm just not sure. I'd have to read it more carefully.
If wait queues stored lists of waiting processes, then notify could involve adding run reasons to all waiting processes. Sounds complicated, but in terms of speed it's probably a lot better than letting those waiting threads remain active.
The way Erlisp uses wait queues, only one process will ever be waiting on it (the one doing the RECEIVE), though many processes can make it wake up (the ones doing SEND). Does that help? Again, I haven't studied CMUCL's threading mechanisms.
Yes, that should make it easier. Unfortunately, I haven't found any good documentation on CMUCL's threading mechanisms, and I don't understand them very well myself. Everything I know about CMUCL threading comes either from reading the compiler source code (which is well commented) or from comparison to Allegro (which CMUCL threading is loosely based on).
Oh BTW, SBCL 0.9.3 will probably break Erlisp, as they've switched from thread IDs to thread objects in the current CVS, as far as I've been told. Should be a rather simple fix though, when the time comes.
It doesn't sound too difficult. Where did you find information on future versions, though?
Mailing list: http://lists.sourceforge.net/lists/listinfo/sbcl-devel
IRC: #lisp at irc.freenode.net
So what's on your schedule now? Threads for more compilers? Process linking? Distributed programming? Just let the mailing list know, and I'll write up a short blog post. (That reminds me, I still need to forward the soc-erlisp e-mails...)
Threads for one more compiler (clisp), then distributed programming.
Fine with me, though you'd then have to do process linking for both parallel /and/ distributed programming afterwards. You may want to work on 'parallel process linking' before working on the distributed programming stuff.
What is process linking?
Process linking is one of Erlang's most important features. Faré said he had discussed this with you. This is from the soc-erlisp e-mail http://parenpower.com/mailman/private/soc-erlisp/2005-July/000015.html: (BTW, why are these archives password protected?)
Yes, Faré explained this to me, in addition to the related topic of error-handling. I forgot the term, though. Thanks for the reminder.
Dirk Gerrits wrote:
As for non-distributed Erlisp, "creating a new thread in the existing Lisp image" is already supported on SBCL 0.9.1, and shouldn't be too hard to port to other Lisps and upcoming SBCL releases. There is still at least one very big missing feature though, and that's linking of processes. (See http://dirkgerrits.com/programming/erlisp/roadmap/ under "Error handling".)
Faré wrote:
Linking of processes is a very important feature of the Erlang programming model. The difficulty being that of killing a thread when it's in the middle of some observable side-effect. This require proper locking and/or proper recovery mechanisms (roll back or roll forward). I've discussed the issue with Eric. Basically, we can either refrain from killing outside of safe-points (notably invoked at Erlisp function calls), or we can require processes to hold a lock while they are doing any kind of non-thread-local side-effect.
Do you have any ideas about what the interface should look like for distributed programming? I've seen some confusing material on erlang websites which suggested that an erlang programmer could write a concurrent program without knowing whether it would be run on one computer (threads) or spread over several. Sounds nice but I never figured out how that worked.
I suggest getting a hold of (borrowing / buying second hand) the book Concurrent Programming in Erlang, Second Edition.
I requested this book on inter-library loan today. It will probably take a week or two to arrive. Until then, I should be able to work on lower level issues without it.
Anyway, the way that works is that Erlang processes live on /nodes/. If you send a message to a process on the same node, what will happen is probably similar to what Erlisp does now. If you send a message to a process on a different node, the message will go over the network. The syntax is identical, so that's probably what those websites mean. And you're supposed to assume the semantics are identical too, that is, message passing can fail in either case. In practice I'd think that message passing /actually/ failing is very, very rare unless the receiver is on a different node though. So you still need to design for distributed programming, but when you do, your code can run both locally and distributedly.
So is each Lisp image automatically considered a node? I need to look more closely at how message passing is implemented.
If you look at src/messaging.lisp, you'll see that the mechanisms for this are already there. SEND calls SEND-USING-NODES to do the work, which has the following specializations:
from a remote node Gives the error "Can only send messages from a local process."
to a remote node Gives the error "Distribution is not implemented yet."
to a threaded process on a local node Does the whole mutex/mailbox/wait-queue thing.
You'd want to replace the second one. ;)
- Dirk
Sounds good. It's nice to have a flexible framework to work in. I need to do some more exploring to figure out what everything does, though.
Eric
Eric Lavigne wrote:
The way Erlisp uses wait queues, only one process will ever be waiting on it (the one doing the RECEIVE), though many processes can make it wake up (the ones doing SEND). Does that help? Again, I haven't studied CMUCL's threading mechanisms.
Yes, that should make it easier. Unfortunately, I haven't found any good documentation on CMUCL's threading mechanisms, and I don't understand them very well myself. Everything I know about CMUCL threading comes either from reading the compiler source code (which is well commented) or from comparison to Allegro (which CMUCL threading is loosely based on).
Okay, hopefully we can get something non-polling for CMUCL like we have now for SBCL. Perhaps the mailing lists can be of some help? (http://www.cons.org/cmucl/support.html)
Process linking is one of Erlang's most important features.
Yes, Faré explained this to me, in addition to the related topic of error-handling. I forgot the term, though. Thanks for the reminder.
Okay, and you still want to proceed with adding distributed programming first before you start tackling this? Fine with me either way, I'm just curious.
I suggest getting a hold of (borrowing / buying second hand) the book Concurrent Programming in Erlang, Second Edition.
I requested this book on inter-library loan today. It will probably take a week or two to arrive. Until then, I should be able to work on lower level issues without it.
Okay great.
Anyway, the way that works is that Erlang processes live on /nodes/. If you send a message to a process on the same node, what will happen is probably similar to what Erlisp does now. If you send a message to a process on a different node, the message will go over the network. The syntax is identical, so that's probably what those websites mean. And you're supposed to assume the semantics are identical too, that is, message passing can fail in either case. In practice I'd think that message passing /actually/ failing is very, very rare unless the receiver is on a different node though. So you still need to design for distributed programming, but when you do, your code can run both locally and distributedly.
So is each Lisp image automatically considered a node? I need to look more closely at how message passing is implemented.
Good question. I would guess so, as they don't share a Lisp memory space. And I think this is true for multiple Erlang instances running on one machine as well.
If you look at src/messaging.lisp, you'll see that the mechanisms for this are already there. SEND calls SEND-USING-NODES to do the work[...]
Sounds good. It's nice to have a flexible framework to work in. I need to do some more exploring to figure out what everything does, though.
Yeah generic functions are great tools for stuff like this. But the framework is experimental anyway, so if you see the need to improve it feel free to bring that up.
- Dirk
Dirk:
I suggest getting a hold of (borrowing / buying second hand) the book Concurrent Programming in Erlang, Second Edition.
Fortunately the first part of the book, which introduces the language concepts, is available as pdf at http://erlang.org/doc.html. The second part, consisting of example applications, is not.
- Willem
I suggest getting a hold of (borrowing / buying second hand) the book Concurrent Programming in Erlang, Second Edition.
Fortunately the first part of the book, which introduces the language concepts, is available as pdf at http://erlang.org/doc.html. The second part, consisting of example applications, is not.
- Willem
Excellent. I still prefer a hard copy so I can get away from the screen on occasion, but this will save me some weeks of waiting.
Thanks, Eric
Willem Broekema wrote:
Dirk:
I suggest getting a hold of (borrowing / buying second hand) the book Concurrent Programming in Erlang, Second Edition.
Fortunately the first part of the book, which introduces the language concepts, is available as pdf at http://erlang.org/doc.html. The second part, consisting of example applications, is not.
Oh cool. Good to know. *updates URL at http://dirkgerrits.com/programming/erlisp/references/*.
- Dirk
Process linking is one of Erlang's most important features.
Yes, Faré explained this to me, in addition to the related topic of error-handling. I forgot the term, though. Thanks for the reminder.
Okay, and you still want to proceed with adding distributed programming first before you start tackling this? Fine with me either way, I'm just curious.
Yes, I just feel more comfortable working on the low-level stuff first.
Eric
-
Dirk Gerrits -
Eric Lavigne -
Willem Broekema