Thanks for the clarification, Doug. I see that your opinion is more nuanced than the version I put up. I'll update it soon and put up a more accurate version of your opinion.
Rich Hickey's ant simulation is a good case study for Clojure concurrency in action. You may find it interesting to read through:
http://clojure.googlegroups.com/web/ants.clj
It will take me some time to fully work through your arguments, as I haven't worked very much with concurrency myself. How do you feel your opinions hold up on the issue of simple convenience for the programmer? I mean, using select() for IO, fork() for concurrency, and Berkeley DB for transactions throughout your programs takes extra effort as compared to using the built-in language mechanisms. In the case of the ant colony simulation, you'd have to worry about the problem of distributing the ants among N forked processes, where N is the number of processors, whereas if you use shared-memory threads, you let the OS handle that.
Regards, Vish Singh
On Mon, Apr 13, 2009 at 7:20 PM, doug@hcsw.org wrote:
We discussed Clojure's concurrency mechanism (Software Transactional Memory).. Doug wonders whether it is overkill to include such a complex mechanism built-in to the language. Much disagreement here among the members.
I have never used Clojure so I can't comment much about it specifically, but that is not what I said about concurrency. Let me elaborate again:
Before you use threads (by which I mean pre-emptively scheduled, shared memory processes) ask yourself why:
- To handle multiple streams of IO? Solution: Schedule one
process to handle all the IO by using an event API like select(), non-pre-emptively scheduled threads like GNU pth (http://www.gnu.org/software/pth/), or "coroutines", "green threads", whatever you wanna call em.
- To take advantage of multiple CPUs or multiple disks?
Solution: Learn how the fork() system call works and use sockets or pipes for message passing.
To me there is no reason to ruin a great language like Common Lisp with thread-related non-determinism. I think having to ban mutable data structures from your language as in Clojure is a symptom of a poor concurrency strategy. As I said at the meeting, in my opinion the best implementation of transactional memory is Berkeley DB:
- Clojure's memory isn't persistent so if your application
crashes or the power goes out you will lose all your memory. BDB can persist the memory to disk guaranteeing transactional consistency (although this is optional).
- As far as I know, Clojure transactional memory can only be
accessed by threads running in the same process. BDB DBs are multi-process AND multi-thread.
- BDB allows replication so that the memory is distributed to
physically separate machines. Each will have a transactionally consistent view of the data.
- BDB DBs can be larger than your address space on 32 bit systems.
I'd guess that Clojure's transactional storage is limited on such systems.
- BDB hotbackups let you snapshot a DB without interrupting
anything. I use the included db_hotbackup utility.
- BDB DBs can be accessed by programs in any language: Perl,
Common Lisp, C, etc. Probably from Clojure too.
toronto-lisp mailing list toronto-lisp@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/toronto-lisp