- erlisp-devel@lists.common-lisp.net - mailman3.common-lisp.net

[erlisp-devel] New Unpaid Item Message from tany70 : #294653688662
by eBay＠common-lisp.net 18 Oct '06

18 Oct '06

1 0

[erlisp-devel] Pong server example
by Geoff Cant 14 Dec '05

14 Dec '05

Hi all, I've written a few little patches and a pong server example for erlisp. My public darcs repo lives at (http://darcs.lisp.geek.nz/~nem/erlisp/) I'm working on linked processes at the moment (and investigating ecl support, distribution and maybe faster openmcl with-timeout replacements). Cheers, -- Geoff ps: sorry for the short/non-explanatory message - it's 23:00ish here :)

1 0

[erlisp-devel] Various
by Faré 22 Nov '05

22 Nov '05

Grid Computing with clisp: http://grid.bmk.com.au/ Also, here's some SBCL code that implements a the basics required for properly handling user-level asynchronous signals. Notably lacking is integration into the event loop: calling (process-check-signals) before to receive things, and after being woken up from sleep if nothing was available at first. (defun make-lock (name) (sb-thread:make-mutex :name name)) (defmacro with-lock-held ((lock &optional whostate) &body body) (declare (ignore whostate)) `(sb-thread:with-mutex (,lock) ,@body)) (defclass process () ((name :initform "" :initarg :name :type string :accessor process-name) (thread :initform nil :initarg :thread :accessor process-thread) (function :initform nil :initarg :function :accessor process-function) (lock :initform (make-lock "thread lock") :accessor process-lock) (signal :initform nil :accessor process-signal) (state :initform nil :initarg :state :accessor process-state))) (defun make-process (name &rest initargs) (apply #'make-instance 'process :name name initargs)) (defvar *current-process* (make-process "Initial process" :thread sb-thread:*current-thread*)) (defun current-process () *current-process*) (defun start-process (function &optional (process-name "Generic Process")) (let ((proc (make-process process-name :function function))) (setf (process-thread proc) (sb-thread:make-thread (process-starter proc))) proc)) (defun process-starter (process) #'(lambda () (let ((*current-process* process)) (loop for termination-reason = (catch 'process-termination (process-check-signals) (funcall (process-function process)) :end) do (ecase termination-reason (:reset) ;;; run the function once again ((:end :kill) (return))))))) ;;; be done (defun process-check-signals () (let ((sig nil)) (with-lock-held ((process-lock *current-process*)) (rotatef sig (process-signal *current-process*))) (when sig (throw 'process-termination sig)))) (defun process-send-signal (process signal) (ecase signal ((:reset :kill) (with-lock-held ((process-lock process)) (let ((sig (process-signal process))) (setf (process-signal process) (max-process-signal sig signal))))) (:kill-on-the-spot ;; --- never do that you're sure the process is in a safe state (sb-thread:terminate-thread (process-thread process)))) (values)) (defconstant-equal +process-signals+ '(:reset :kill :kill-on-the-spot)) (defun max-process-signal (sig1 sig2) (if (member sig1 (member sig2 +process-signals+)) sig1 sig2)) ;; A process will only handle an asynchronous signal at a safe point, ;; as declared by said process calling PROCESS-CHECK-SIGNALS. ;; Any attempt at killing a process when it isn't explicitly checking signals ;; is *UNSAFE*, unless it is somehow _guaranteed_ that the process ;; isn't holding any lock, isn't modifying some shared data-structure, ;; and including the heap meta-data (i.e. you lose if you kill a thread in the ;; middle of consing or otherwise while it is disabling garbage-collection). ;; This is valid for *any* Lisp implementation, including CMUCL and whatelse ;; (except that CMUCL's green thread model might avoid the CONSing/GC ;; part of the problem). Use Erlang for a language without this problem at all ;; (because its processes can't do any side-effect or sharing except ;; through its system-managed atomic message passing primitives). ;; See the mess that was thread.interrupt in Java and why it was discontinued. ;; We sorely miss some kind of application-extensible PCLSRing. -- fare ;; ;; GOLDEN RULE: thou shall not stop or interrupt a thread in any language ;; that uses any kind of shared memory. EVER. Because you may catch the thread ;; while it has its pants down, and then you're in deep trouble. ;; ;; BOTTOM LINE: if any process it meant to receive any of asynchronous signal ;; through PROCESS-PRESET, PROCESS-RESET or PROCESS-KILL, then it *must* ;; poll for signals with PROCESS-CHECK-SIGNALS at regular safe points. ;; (e.g. in its event loop). ;; ;; Note: if that was ever needed, we could possibly have a "safe to kill" flag ;; in the process structure, that the thread could raise when it's in a safe ;; mode with no locking, no data sharing, no consing. Then, signal senders ;; could check this flag, and if present, scrap the whole thread, and start ;; a new one if a function is to be (re)started. (defun process-preset (process function &rest args) (setf (process-function process) #'(lambda () (apply function args))) (process-reset process)) (defun process-reset (process) (assert (process-function process)) (process-send-signal process :reset)) ;; Words of wisdom from the SBCL source code: ;; A moderate degree of care is expected for use of interrupt-thread, ;; due to its nature: if you interrupt a thread that was holding ;; important locks then do something that turns out to need those ;; locks, you probably won't like the effect. (defun process-interrupt (process function) (when (process-thread process) (sb-thread:interrupt-thread (process-thread process) function))) (defun process-kill (process) (process-send-signal process :kill)) [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Government's view of the economy could be summed up in a few short phrases : If it moves, tax it. If it keeps moving, regulate it. And if it stops moving, subsidize it. -- Ronald Reagan (1986)

1 0

[erlisp-devel] Fwd: [Sbcl-devel] async signals
by Faré 20 Sep '05

20 Sep '05

I found this very interesting discussion on sbcl-devel. It shows the difficulty of doing things right if we are to kill processes asynchronously, yet they want to do things properly wrt catching them, ensuring atomicity of memory operations, freeing resources (notably alien data), etc. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Perhaps those of us who care about quality programs have not spoken up often enough -- `for bad programs to triumph requires only that good programmers remain silent.' I call this passivity the `Silence of the Lambdas.' -- hbaker ---------- Forwarded message ---------- From: Gábor Melis <mega(a)hotpop.com> Date: 30-Aug-2005 06:15 Subject: Re: [Sbcl-devel] async signals To: Alexander Kjeldaas <astor(a)fast.no> Cc: sbcl-devel(a)lists.sourceforge.net On Monday 22 August 2005 22:23, Alexander Kjeldaas wrote: > OPEN, CLOSE and other functions that need cleanup could assert that > *INTERRUPTS-ENABLED* is NIL. Thus unclean code could be weeded out. > > Maybe something like this: > > (let (x) > (unwind-protect-without-interrupts-in-cleanup > (progn > (setq-without-interrupts x (open ...)) > ...) > (when x (close x))))) > > would be easier to read? > > astor Yes, it's easier to read and within the bowels of sbcl it might even be utilized. Since async signals are not standard, people just don't expect arbitrary things to happen at any time. It's just hard. Something must be done to bend async signals into lisp. I thought about making serve-event check for pending interrupts. When a possibly blocking I/O call is made the user should be well prepared to handle arbitrary conditions anyway, right? Yeah, it is a bit of stretch. Especially considering that the caller needs to know exactly which calls can do I/O or call serve-event. Not likely, around methods and such make it even less so. At this point I got disheartened and come to think that abandoning a side-effecting computation (with-timeout, terminate-thread, C-c + abort) is practically unsafe wrt unwind-protect. With the exception of with-timeout they should not "normally" (not during development or as last resort) be used. Ordinary code is just full of assumptions of what can and cannot fail. That leaves us with some documentation to write and with-timeout. A quick audit on the paserve code base revealed that its need could be satisfied with: - deadlines on streams Currently, timeouts are per-operation. Sending one byte every timeout seconds is enough to keep a reader on the other end of the stream busy. For a web server, that has a stream for each request but performs multiple operations on the stream an upper limit for the sum of the duration of those operations is needed. (setf (stream-deadline stream) (+ 180 (get-universal-time))) (loop repeat 100 do (read-char stream)) A slight problem is clobbering deadlines already set. The following macro temporarily overwrites the deadline if the new deadline is earlier than the old one. (with-stream-deadline (stream (+ 5 (get-universal-time))) (read stream)) - a timeout on socket-connect, open (?), close The current implementation of with-timeout should work for aborting the system calls (connect, open). That would mean adding a :timeout parameter to socket-connect, and open on the lisp side. close with :abort t should not block, unless SO_LINGER is set. The plan: 1) implement a with-timeout that works with threads 2) add timeout parameters to socket-connect, open, ... (?) 3) implement stream deadlines Unfortunately, 1) needs a scheduler (like Zach's timer http://www.cliki.net/TIMER, 500 lines), but - on the bright side - it's still possible to implement with setitimer and does not require a dedicated thread. Please, flame this approach properly. Cheers, Gábor ------------------------------------------------------- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf _______________________________________________ Sbcl-devel mailing list Sbcl-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel

3 9

[erlisp-devel] broken process linking code
by Eric Lavigne 16 Sep '05

16 Sep '05

This is the process-linking code I wrote over the summer. It doesn't work, and classes keep me too busy to continue working on it right now. Hopefully someone will have time to fix it. Otherwise I will probably get back to it in a few months. The problem, most likely, is in messaging.lisp. I added calls to a new function at the beginning of send, receive, and receive-with-matcher, so that if a process were marked for destruction it would die before affecting other processes (thus the hook in send) and as soon as possible (thus the hooks in a couple other places). I didn't understand the macros that I was trying to modify, so I was not so surprised to get a bad result. Keeping the hook only in send might fix the problem, but that could be horribly inefficient depending on how often the process tries to send messages. There is also a need for a new test for process-linking. I vaguely remember writing at least a basic test, but there doesn't seem to be one in my working directory. manager.lisp most new code messaging.lisp hooks into old code created in send/receive/receive-with-matcher process.lisp "time-to-die" flag added to process class erlisp.asd new entry for manager.lisp

1 0

[erlisp-devel] Fwd: [Sbcl-devel] async signals
by Faré 15 Sep '05

15 Sep '05

Next message by Gabor on same thread... [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] In reviewing the history of the English Government, its wars and its taxes, a bystander, not blinded by prejudice nor warped by interest, would declare that taxes were not raised to carry on wars, but that wars were raised to carry on taxes. -- Thomas Paine, "Rights of Man" ---------- Forwarded message ---------- From: Gábor Melis <mega(a)hotpop.com> Date: 15-Sep-2005 11:08 Subject: Re: [Sbcl-devel] async signals To: sbcl-devel(a)lists.sourceforge.net Time flies. With #lisp becoming immune to async unwind, it's now sbcl-devil's time to think. The following might become part of the internals manual if the patch goes in, hence the repetition from earlier posts. * Issues with asynchronous unwinds (AUs) Consider the following example: (with-timeout 1 (let (fd) (unwind-protect (progn (setq fd (open ...)) ...) (when fd (close fd))))) There are several ways things can go wrong: ** (SETQ FD (OPEN ...)) can be interrupted and unwound just after OPEN returns but before SETQ is done ** foreign code may not like being aborted: imagine a Berkeley db update call in the second ..., it will not like being unwound in the middle ** cleanups may not be run: if the cleanup is interrupted it can be unwound without closing the fd ** AUs can clobber each other Suppose we have a thread that can be terminated cleanly: (defun greet () (unwind-protect (loop (write-line "Hello, world") (sleep 1)) (write-line "Good bye"))) (let ((thread (make-thread #'greet))) (sleep (random 6)) (terminate-thread thread)) So far so good. But what if two other threads try to terminate it at the same time? The second terminate can hit while the first unwinding is in still progress. This is not a problem since the target of the two throws is the same. Now, what happens if there is another AU with a different target? (defun greet-impatiently () (handler-case (sb-ext:with-timeout 3 (greet)) (sb-ext:timeout ()))) Let's try terminating it: (let ((thread (make-thread #'greet-impatiently))) (sleep (random 6)) (terminate-thread thread)) There are several possible outcomes, but the most interesting is when TERMINATE-THREAD starts unwinding then the timeout hits does another NLX to the HANDLER-CASE and the thread termination request is lost. Note that the UNWIND-PROTECT in GREET is not strictly needed for this scenario to occur. In general it is very hard to write reliable programs when multiple AUs play and can cancel or steer away (see http://www.lisp.org/HyperSpec/Issues/iss152-writeup.html) an ongoing unwind. * Implementation ** Interruption By definition an interruption is a function that is invoked asynchronously. ** AU unsafe zone A thread is said to be in AU unsafe zone iff execution is within a WITHOUT-ASYNCHRONOUS-UNWIND form, an UNWIND-PROTECT cleanup or it's unwinding. Note that there can be multiple simultaneous unwinds: (catch 'aaa (unwind-protect (throw 'aaa 'a) (catch 'bbb (throw 'bbb 'b)))) The implementation (on x86 only for the time being) keeps track of the outermost unwind-protect block that's being unwound to. If an AU occurs in an unsafe zone then the current AU handlers get to decide what happens. ** AU handlers Interruptions are run in a pretty normal dynamic environment with all the condition handlers that were setup in the thread. When an NLX occurs that leaves the interruption UNSAFE-ASYNCHRONOUS-UNWIND (a SERIOUS-CONDITION) is signalled if the interrupted thread is in an unsafe zone. Now if this condition was handled by the normal condition handlers a simple HANDLER-CASE for SERIOUS-CONDITION could unknowingly unwind the stack when it's unsafe. Hence, UNSAFE-ASYNCHRONOUS-UNWIND is signalled with a different set of condition handlers active. These handlers can be established by ASYNCHRONOUS-UNWIND-HANDLER-BIND or from within the interruption with PUSH-INTERRUPTION-UNWIND-HANDLERS, POP-INTERRUPTION-UNWIND-HANDLERS, the setfable INTERRUPTION-UNWIND-HANDLERS. (Bleh. I can see no way around this.) These handlers are run, protected by a WITHOUT-INTERRUPTS, when the interruption is about to be left (from an UNWIND-PROTECT cleanup around the interruption, in fact). The *only* restarts available are RETRY-LATER, ABORT (aborts the unwind) and CONTINUE (forces the unsafe unwind to continue). The AU handler may do an NLX by any of the usual suspects (THROW, RETURN-FROM, GO), but if it signals a condition only the AU handlers are there to help. Behind the scenes interruptions are run by INVOKE-INTERRUPTION that on a NLX from the interruption signals an UNSAFE-ASYNCHRONOUS-UNWIND (a SERIOUS-CONDITION). ** The RETRY-LATER restart The default AU handler invokes this restart. This is similar to the ABORT restart, but it sets up a timer to run the whole interruption again in a short time (~0.1s, randomized). Alternatively the implementation could poll for interruptions when the unsafe zone is potentially exited. This is not done since detecting unsafe zones is currently expensive, because the stack is searched for cleanup frames. ** Foreign code Foreign code is to be wrapped in WITHOUT-ASYNCHRONOUS-UNWIND by default. System calls are a different: they can be interrupted without ill effects. ** Examples Consider the slightly modified example: (with-timeout 1 (let (fd) (unwind-protect (progn (without-asynchronous-unwind () (setq fd (open ...))) ...) (when fd (close fd))))) This is bullet-proof wrt to open/close. The problems outlined above are gone: the fd cannot be lost by unwinding just after OPEN returns but before the SETQ is completed; the cleanup and CLOSE within it is guaranteed to run without much disturbance. But an unwind due to timeout cannot happen in the AU unsafe zones, and that means OPEN and maybe CLOSE should take a timeout argument. Also note that WITH-TIMEOUT doesn't work in cleanup forms (!), so this will not return in one second if CLOSE runs into problems: (unwind-protect ... (with-timeout 1 (close fd))) One can work around that by: (unwind-protect ... (asynchronous-unwind-handler-bind ((unsafe-asynchronous-unwind #'continue)) (with-timeout 1 (close fd)))) but more care is needed as it allows an ongoing unwind to be lost. * An alternative: double cross implementation It can be argued that an AU becomes a problem only if it would cross a WITHOUT-ASYNCHRONOUS-UNWIND, a cleanup, or cancel/steer away an ongoing unwind (i.e. it crosses an border of an interruption and an unsafe zone delimiter). Ultimately, it is about user expectations about what can go wrong and when. It's unclear if such a weakening of semantic the guarantees is worthwhile. In the above example if CLOSE has a HANDLER-CASE for ERROR then an AU can still make it fail without closing the fd. * TODO ** detection of cleanups is slow (walks the stack) ** detection of cleanups is racy And here is the patch: http://retes.hu/~mega/au.patch It has just reached a state where one can experiment with C-c-ing: (without-asynchronous-unwind () (sleep 5)) (unwind-protect t (sleep 5)) and get reasonable behaviour, so it must be ready for comments. Cheers, Gábor ------------------------------------------------------- SF.Net email is sponsored by: Tame your development challenges with Apache's Geronimo App Server. Download it for free - -and be entered to win a 42" plasma tv or your very own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php _______________________________________________ Sbcl-devel mailing list Sbcl-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel

1 0

[erlisp-devel] Erlisp file loading order
by Eric Lavigne 02 Sep '05

02 Sep '05

I am running into another issue involving file loading order. node.lisp contains the following line: (defvar *current-node* (make-instance 'local-node)) Now that nodes are more than just an empty class (each node has a system-manager), there is a need to call functions while creating node instances (such as spawn). Is it okay to have circular dependencies in asdf? (:file "node" :depends-on ("package" "process")) (:file "process" :depends-on ("package" "mailbox" "node" "compatibility"))

3 2

[erlisp-devel] undefined-symbol
by Eric Lavigne 02 Sep '05

02 Sep '05

I am confused by this error message. ; ; File: /root/erlisp/erlisp-cmucl/erlisp/src/manager.lisp ; In: DEFUN SYSTEM-MANAGER-PROCESS ; (RECEIVE-WITH-MATCHER (COND-MATCHER M) (T M)) ; Error: (during macroexpansion) ; Error in KERNEL::UNDEFINED-SYMBOL-ERROR-HANDLER: the function LISTEN-TO-SYSTEM-MANAGER is undefined. ; ; Error: (during macroexpansion) ; Error in KERNEL::UNDEFINED-SYMBOL-ERROR-HANDLER: the function LISTEN-TO-SYSTEM-MANAGER is undefined. ; ; Converted SYSTEM-MANAGER-PROCESS. It complains that function LISTEN-TO-SYSTEM-MANAGER is undefined, but you can see here that this undefined function is defined immediately before the function that complains about it being missing. It seems like everything I've tried over the last two days lead to some sort of missing function error, and this still doesn't make sense to me. When I write small Lisp codes on my own, it doesn't seem to matter in which order I define functions. As long as all the functions are defined before I start actually running the functions, everything just works. What is different here? Is symbol scope related to Excerpt from manager.lisp: (defun listen-to-system-manager () (when (slot-value (current-process) 'time-to-die) (suicide "System manager requested termination."))) (defun system-manager-process () (do ((msg (receive-with-matcher (cond-matcher m) (t m)) (receive-with-matcher (cond-matcher m) (t m)))) (nil) (cond ((eq 'link (first msg)) (setf (gethash (list (second msg) (third msg)) *link-table*) (fourth msg))) ((eq 'exit (first msg)) (maphash #'(lambda (k v) (let ((src (second msg)) (reason (third msg))) (when (member src k) (when (eq src (first k)) (cond ((or (eq v 'notify) (and (eq v 'default) (eq reason 'normal))) (send (second k) msg)) (t (kill-process (second k))))) (remhash k *link-table*)))) *link-table*))))) For reference, I have enclosed manager.lisp (new) and process.lisp (modified). Eric

3 3

[erlisp-devel] receive
by Eric Lavigne 02 Sep '05

02 Sep '05

(setf msg (receive t)) Would this receive any available message and assign that message to msg? Eric

2 2

[erlisp-devel] Description of pushed patches.
by Dirk Gerrits 01 Sep '05

01 Sep '05

sbcl no longer supports (sb-thread:current-thread-id). bryan d. o'connor <bryan(a)lunch.org>**20050830023327] openmcl-threads using semaphores. bryan d. o'connor <bryan(a)lunch.org>**20050901181244] allegro-threads using gates. bryan d. o'connor <bryan(a)lunch.org>**20050901235948 patch written by Dirk Gerrits.

1 0