[Bordeaux-threads-devel] Deprecating recursive locks
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether. What do you think ? -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
I think it is a very good idea. When are you planning the 1.0 release? It would be nice to go through all the systems dependent on bordeaux-threads in Quicklisp and notify the maintainers of any that use recursive locks to fix their code. Vladimir On Tue, Apr 17, 2012 at 11:21 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
-- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
_______________________________________________ Bordeaux-threads-devel mailing list Bordeaux-threads-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/bordeaux-threads-devel
On Tue, 2012-04-17 at 12:14 -0400, Vladimir Sedach wrote:
I think it is a very good idea.
When are you planning the 1.0 release? It would be nice to go through all the systems dependent on bordeaux-threads in Quicklisp and notify the maintainers of any that use recursive locks to fix their code.
A month or two, perhaps. It depends on my spare time -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
17.04.2012, 20:14, "Vladimir Sedach" <vsedach@gmail.com>:
I think it is a very good idea.
When are you planning the 1.0 release? It would be nice to go through all the systems dependent on bordeaux-threads in Quicklisp and notify the maintainers of any that use recursive locks to fix their code.
If the public API is changed in a not-backward-compatible way, I would suggest to release new ASDF system (bordeaux-threads2), and leave the old version available in quicklisp forever. People who are ready to use new version, just add bordeaux-thread2 into their ASDF dependency. Others are able to use old version. This is a good practice. And it costs nothing. Some links on this subject: http://lispcaveats.tumblr.com/post/13259176455/ffi-linking-against-shared-li... http://semver.org/
If the public API is changed in a not-backward-compatible way, I would suggest to release new ASDF system (bordeaux-threads2), and leave the old version available in quicklisp forever.
People who are ready to use new version, just add bordeaux-thread2 into their ASDF dependency. Others are able to use old version.
This is a good practice. And it costs nothing.
IMHO it's not a good practice and it does cost complexity/noise. you lose the clarity around the digital representation(s) of the identity of the library. there's only one b-t library, and the fact that it has changed in a way that made it incompatible with something else does not change its identity. but it's only the 0.02 of an outsider... -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 OTR XMPP: 8647EEAC EA30FEEF E1B55146 573E52EE 21B1FF06
On Wed, 2012-04-18 at 01:48 +0400, Anton Vodonosov wrote:
17.04.2012, 20:14, "Vladimir Sedach" <vsedach@gmail.com>:
I think it is a very good idea.
When are you planning the 1.0 release? It would be nice to go through all the systems dependent on bordeaux-threads in Quicklisp and notify the maintainers of any that use recursive locks to fix their code.
If the public API is changed in a not-backward-compatible way, I would suggest to release new ASDF system (bordeaux-threads2), and leave the old version available in quicklisp forever.
People who are ready to use new version, just add bordeaux-thread2 into their ASDF dependency. Others are able to use old version.
This is a good practice. And it costs nothing.
Some links on this subject: http://lispcaveats.tumblr.com/post/13259176455/ffi-linking-against-shared-li...
That post contains a good analysis of the problem, but then gives the worst possible advice to solve it
Point 5 says "Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable". So no problem there -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
17.04.2012, 19:21, "Stelian Ionescu" <sionescu@cddr.org>:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
Does it mean that every time, when trying to lock an object, we will need to ensure it hasn't been locked already? If yes, it complicates writing synchronization code. But if it can not be reliably supported based on the features provided by all lisps, then removing them may probably be right.
On Tue, Apr 17, 2012 at 11:21 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
I currently use a recursive lock for debug logging. In the past I wrapped *debugger-hook* with a recursive lock in order to avoid multiple simultaneous debugger popups. Both are cases of throwaway code. David Butenhof does mention that recursive locks "can be a great tool" as long as they are properly understood as a temporary measure. That's how I use them, and others may as well. Yes, they are a hack, but they are a convenient hack. If recursive locks are removed then I would expect people to write their own half-ass implementations, as I would do for my debug logger. I agree that using them in a non-temporary context may constitute poor design, however that does not imply that they should be removed. Perhaps adding a note to the documentation regarding the hackiness of recursive locks (and/or even linking to the Butenhof post) would suffice in lieu of outright deletion? If you do decide to remove them, consider issuing deprecation warnings in the next release (or longer) before the actual removal. There is at least one logging library in quicklisp which uses a recursive lock. I assume other projects out there would be broken as well.
On Wed, 2012-05-02 at 11:15 -0400, James M. Lawrence wrote:
On Tue, Apr 17, 2012 at 11:21 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
I currently use a recursive lock for debug logging. In the past I wrapped *debugger-hook* with a recursive lock in order to avoid multiple simultaneous debugger popups. Both are cases of throwaway code.
David Butenhof does mention that recursive locks "can be a great tool" as long as they are properly understood as a temporary measure. That's how I use them, and others may as well. Yes, they are a hack, but they are a convenient hack.
If recursive locks are removed then I would expect people to write their own half-ass implementations, as I would do for my debug logger. I agree that using them in a non-temporary context may constitute poor design, however that does not imply that they should be removed.
Perhaps adding a note to the documentation regarding the hackiness of recursive locks (and/or even linking to the Butenhof post) would suffice in lieu of outright deletion?
If you do decide to remove them, consider issuing deprecation warnings in the next release (or longer) before the actual removal. There is at least one logging library in quicklisp which uses a recursive lock. I assume other projects out there would be broken as well.
Which one ? Such library is broken because few CL implementations actually have recursive locks -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
On Wed, May 2, 2012 at 11:27 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
On Wed, 2012-05-02 at 11:15 -0400, James M. Lawrence wrote:
On Tue, Apr 17, 2012 at 11:21 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
I currently use a recursive lock for debug logging. In the past I wrapped *debugger-hook* with a recursive lock in order to avoid multiple simultaneous debugger popups. Both are cases of throwaway code.
David Butenhof does mention that recursive locks "can be a great tool" as long as they are properly understood as a temporary measure. That's how I use them, and others may as well. Yes, they are a hack, but they are a convenient hack.
If recursive locks are removed then I would expect people to write their own half-ass implementations, as I would do for my debug logger. I agree that using them in a non-temporary context may constitute poor design, however that does not imply that they should be removed.
Perhaps adding a note to the documentation regarding the hackiness of recursive locks (and/or even linking to the Butenhof post) would suffice in lieu of outright deletion?
If you do decide to remove them, consider issuing deprecation warnings in the next release (or longer) before the actual removal. There is at least one logging library in quicklisp which uses a recursive lock. I assume other projects out there would be broken as well.
Which one ? Such library is broken because few CL implementations actually have recursive locks
log4cl, released a few months ago. You mentioned that Clozure does not support recursive locks. The funny thing about Clozure is that ccl:make-lock actually returns a recursive lock. So bt:make-recursive-lock is, through coincidence, providing a recursive lock by calling ccl:make-lock. Thus users may be fooled into thinking bordeaux-threads is making some guarantee when it hasn't. The behavior of Clozure may change in the future, for all we know. The best of both worlds is to move recursive locks to the introspection/debugging section of the documentation. Recursive locks would be there as convenience for the use case I mentioned, while users would be made aware that they are not dependable across implementations.
On Wed, 2012-05-02 at 13:13 -0400, James M. Lawrence wrote:
On Wed, May 2, 2012 at 11:27 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
On Wed, 2012-05-02 at 11:15 -0400, James M. Lawrence wrote:
On Tue, Apr 17, 2012 at 11:21 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
I currently use a recursive lock for debug logging. In the past I wrapped *debugger-hook* with a recursive lock in order to avoid multiple simultaneous debugger popups. Both are cases of throwaway code.
David Butenhof does mention that recursive locks "can be a great tool" as long as they are properly understood as a temporary measure. That's how I use them, and others may as well. Yes, they are a hack, but they are a convenient hack.
If recursive locks are removed then I would expect people to write their own half-ass implementations, as I would do for my debug logger. I agree that using them in a non-temporary context may constitute poor design, however that does not imply that they should be removed.
Perhaps adding a note to the documentation regarding the hackiness of recursive locks (and/or even linking to the Butenhof post) would suffice in lieu of outright deletion?
If you do decide to remove them, consider issuing deprecation warnings in the next release (or longer) before the actual removal. There is at least one logging library in quicklisp which uses a recursive lock. I assume other projects out there would be broken as well.
Which one ? Such library is broken because few CL implementations actually have recursive locks
log4cl, released a few months ago.
You mentioned that Clozure does not support recursive locks. The funny thing about Clozure is that ccl:make-lock actually returns a recursive lock.
Indeed I was fooled by that. Good to know, this is a excellent reason not to ever use Clozure -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
On 4/17/2012 8:21 AM, Stelian Ionescu wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
I use CCL at the moment Not having recursive locks makes locking code more complicated. With recursive locks I only need to think about ensuring order when more than one lock is involved. Without recursive lock, every possible call path to every lock needs to be thought through. I confirmed that CCL does support recursive locking http://clozure.com/pipermail/openmcl-devel/2012-May/013536.html FWIW for .net http://msdn.microsoft.com/en-us/library/de0542zz%28v=vs.71%29.aspx "It is legal for the same thread to invoke Enter more than once without it blocking" for java http://docs.oracle.com/javase/tutorial/essential/concurrency/locksync.html "a thread can acquire a lock that it already owns" (so I assume ABCL does recursive locks) I read the message by Dave Butenhof http://groups.google.com/group/comp.programming.threads/msg/d835f2f6ef8aed99?hl=en&pli=1 that I think was pointed as the logic behind getting rid of recursive mutexes in bordeaux. I did a basic read. I am not an expert. I came to the conclusion, that he is confounding having a short duration hold on a lock with having a single lock request within a thread. That is, he is using the call stack depth to imply longer code, or in other words he is coming from the world of large functions. Even though those two things correlate often, that does not meean I can not have a couple of small functions each containing a with-... macro that does locking and then not have to bother with which exact combinations of mutual calls can happen among them. While I agree it would be good if my code worked without recursive locks, I think I'd prefer having recursive locks. I do think of keeping locked duration short, but I don't scratch my head around call sequences within a thread requesting the same lock. Unlike any other aspect of CL, threading in CL is very immature and removing this support is not going to help. I hope the decision to remove support will be reconsidered and some alternative like runtime/compile error will be implemented for implementations that don't support recursive locks.. -Antony
Another argument against recursive locks that hasn't come up yet is that they fail very badly when used with condition variables. I think the fact that recursive locks aren't supported by most implementations plus the fact that they behave differently wrt condition variables on different implementations and operating systems are very good arguments for leaving recursive locks out of a portable threading library. Vladimir On Wed, May 2, 2012 at 3:10 PM, Antony <lisp.linux@gmail.com> wrote:
On 4/17/2012 8:21 AM, Stelian Ionescu wrote:
They're an ugly hack, as described their inventor David Butenhof (http://tinyurl.com/butenhof-recursive-mutexes) and aren't supported by all lisps: Allegro, Clozure, CMUCL, SBCL(it has a kind of recursive lock, but it's not a posix recursive lock), so I was thinking of making a 1.0 release in which to remove them altogether.
What do you think ?
I use CCL at the moment Not having recursive locks makes locking code more complicated. With recursive locks I only need to think about ensuring order when more than one lock is involved. Without recursive lock, every possible call path to every lock needs to be thought through.
I confirmed that CCL does support recursive locking http://clozure.com/pipermail/openmcl-devel/2012-May/013536.html
FWIW for .net http://msdn.microsoft.com/en-us/library/de0542zz%28v=vs.71%29.aspx "It is legal for the same thread to invoke Enter more than once without it blocking"
for java http://docs.oracle.com/javase/tutorial/essential/concurrency/locksync.html "a thread can acquire a lock that it already owns" (so I assume ABCL does recursive locks)
I read the message by Dave Butenhof http://groups.google.com/group/comp.programming.threads/msg/d835f2f6ef8aed99?hl=en&pli=1 that I think was pointed as the logic behind getting rid of recursive mutexes in bordeaux. I did a basic read. I am not an expert. I came to the conclusion, that he is confounding having a short duration hold on a lock with having a single lock request within a thread. That is, he is using the call stack depth to imply longer code, or in other words he is coming from the world of large functions. Even though those two things correlate often, that does not meean I can not have a couple of small functions each containing a with-... macro that does locking and then not have to bother with which exact combinations of mutual calls can happen among them.
While I agree it would be good if my code worked without recursive locks, I think I'd prefer having recursive locks. I do think of keeping locked duration short, but I don't scratch my head around call sequences within a thread requesting the same lock.
Unlike any other aspect of CL, threading in CL is very immature and removing this support is not going to help. I hope the decision to remove support will be reconsidered and some alternative like runtime/compile error will be implemented for implementations that don't support recursive locks..
-Antony
_______________________________________________ Bordeaux-threads-devel mailing list Bordeaux-threads-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/bordeaux-threads-devel
Vladimir Sedach wrote:
Another argument against recursive locks that hasn't come up yet is that they fail very badly when used with condition variables. I think the fact that recursive locks aren't supported by most implementations plus the fact that they behave differently wrt condition variables on different implementations and operating systems are very good arguments for leaving recursive locks out of a portable threading library.
I had not paid much attention to this issue until now but this last argument about condition variables just made such that the default behavior of #'mt:make-lock of MKCL will be from now on to produce a non-recursive lock even if this is a break with the behavior inherited from ECL. This starting with MKCL 1.1.0 RC2. If you want recursive locks you will have to request it explicitly with (make-lock :recursive t). There may be some protest against this change but the strength of the "condition variable" point will override them. Cheers, Jean-Claude Beaudoin P.S.: SLIME blew up big time on this change. I had to force swank-backend:make-lock back into producing recursive locks in order to prevent it from crashing. That's bad... I am also considering whether #'mt:condition-wait should emit a warning if it ever receives a recursive lock. (Even more so if the recursive lock has a count above 1!) What do you think of this?
On Wed, May 2, 2012 at 4:57 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
Another argument against recursive locks that hasn't come up yet is that they fail very badly when used with condition variables. I think the fact that recursive locks aren't supported by most implementations plus the fact that they behave differently wrt condition variables on different implementations and operating systems are very good arguments for leaving recursive locks out of a portable threading library.
Ensuring portability seems outside the scope of bordeaux. Take the example of Clozure, where all locks are recursive locks. Is it bordeaux's responsibility to ensure that the thing returned by make-lock is not locked twice? Indeed bordeaux includes functions which are explicitly unportable. The issue you mention with condition variables is a good reason to update the documentation on recursive locks. Why not simply note the behavior as unportable, as with destroy-thread? Nobody is arguing to remove the five functions which "are not advised to be called from normal user code". They are kept because they are useful. The same is true of recursive locks, which for example are convenient for logging. Isn't that reason enough to keep them? It appears that the 11 implementations covered by bordeaux all support recursive locks except Macintosh CL. All locking is recursive in Clozure and CMUCL. Allegro has (mp:with-process-lock (lock :norecursive nil) ...). I've had no trouble with SBCL's non-POSIX recursive lock, though I only use it for logging.
On 5/2/2012 1:57 PM, Vladimir Sedach wrote: While what you say below I am sure is true, is it good enough to say recursive locking should be removed? Trying to use the same mutex for normal locking and again as part of condition variable structure I think is breaking the abstraction of condition variable. Also, if someone writes code that recursively locks to control access to a resource and then tries to do signalling deep in the call stack using that same mutex combined with a condition var, that's just bad code. Saying we shouldn't have recursive mutexes because of that is like saying we shouldn't have threads cause they can corrupt each others variables. I do like this mail thread since I think it's good to discuss this. Sorry, I did not respond early enough. -Antony
Another argument against recursive locks that hasn't come up yet is that they fail very badly when used with condition variables. I think the fact that recursive locks aren't supported by most implementations plus the fact that they behave differently wrt condition variables on different implementations and operating systems are very good arguments for leaving recursive locks out of a portable threading library.
Vladimir
participants (7)
-
Anton Vodonosov -
Antony -
Attila Lendvai -
James M. Lawrence -
Jean-Claude Beaudoin -
Stelian Ionescu -
Vladimir Sedach