"Hoehle, Joerg-Cyril" Joerg-Cyril.Hoehle@t-systems.com writes:
Therefore I was disappointed by your roadmap which talks a lot about OS threads and special (thread-local) variables.
Sorry to hear that, care to elaborate?
Recently I came again across Lightweight Languages Workshop 2002 http://ll2.ai.mit.edu/ You'll notice that Joe Armstrong's web server starts where Apache has already broken down due to too many concurrent users. I you go with OS threads, you'll never get there.
Well OS threads are just the easiest way that Lisp offers me to implement these abstractions. As I wrote in my roadmap, I see two ways to later improve performance:
* Using continuations as green threads. This would limit the points at which processes can be preempted, but that can be partially alleviated by assigning an operating system thread to every few green threads. Also, less preemption is not necessarily a problem.
I've already done some work on this, but I'm currently swamped in school work.
* Hacking the Common Lisp implementation. If SBCL would have more light-weight threads, then so would Erlisp. Alternatively, Erlisp could be ported to CLISP, where light-weight processes could be added to the virtual machine from scratch. CLISP also has a socket library so porting Erlisp's distribution aspects should be straightforward."
This is something I'd prefer not to do, but I see no way around it if I want to match or surpass Erlang's performance. (And I /do/ want to. I apologize if my website gave you the wrong idea.)
I reread part's of Graham's On Lisp yesterday. There's one section where he mentions that a completely unperformant but working thing is still valuable to start with because it allows experimentation (similar topic idea in the article about the failure of the NIL machine/project).
That's the reason I am using OS threads at all. On Lisp's CPS macros are just not satisfactory, and a CPS code walker is more work than using an OS thread library. So I'm going to get something up and running with threads first (roadmap step 1) and then make it fast with a CPS code walker / CL implementation hacking / whatever (roadmap step 3). In fact, I think I might swap step 2 and 3 as performance in parallelism is currently more important to me than distribution.
The reason why I want to depart from OS threads is because I expect different ways of computing arise when "threads" are extremely light weight (as in Erlang or Oz, not in UNIX), like different ways of computing arose when stack space became cheap or when memory became cheap.
I am convinced of that as well. Otherwise I wouldn't see a real need for Erlisp, as a standard OS threading API would have been enough.
I.e. these days, many GNU programs allocate a few MB on the stack without shame, and programs long do in memory only what once were expensive disk-bound operations. This allows different algorithms to be used to address problems.
OTOH maybe one can wait until a typical Linux can fork a 300000 threads on a 256MB machine (I haven't checked how far StateThreads and the other approaches can go). That's ~1KB per thread!
That would be a great thing for the GNU/Linux crowd, myself included, but it wouldn't really solve anything for the rest of the world.
Cf. http://c2.com/cgi/wiki?MessagingAsAlternativeToMultiThreading
Thanks for the reference, I'll check it out.
PS: I didn't see a comp.lang.lisp thread, nor a wiki/message forum on your site and I still think eurolisp is not the right forum for this.
I did post at comp.lang.lisp: http://groups.google.com/groups?selm=87acurudww.fsf%40dirkgerrits.com&rn... but there have been absolutely no replies there.
I agree that a better place for discussion is needed. How about a common-lisp.net mailing list?
Kind regards,
Dirk Gerrits