Loading "lparallel" hangs
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
Hello, can you provide me some hints for the following problem ? I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed. Regards, Robert Larice
I tried to understand where the issue is located. In :lparallel file lparallel-20160825-git/src/thread-util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux-threads:condition-wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux-threads:condition-wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue: (ql:quickload :bordeaux-threads) (let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001))) I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever. Can you help me to understand this better and to work around it ? Regards Robert Larice Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Hey Robert, I can't reproduce the problem with the newest ECL code (from git develop branch) on my host. lparallel works fine and this snippet does terminate after timeout. CL-USER> (ext:lisp-implementation-vcs-id) "ba6e6ddde780c097918673f16c7aba05f354d022" Best regards, Daniel W dniu nie, 02.09.2018 o godzinie 12∶27 +0200, użytkownik Robert Larice napisał:
I tried to understand where the issue is located.
In :lparallel file lparallel-20160825-git/src/thread-util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux-threads:condition-wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux-threads:condition-wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue:
(ql:quickload :bordeaux-threads)
(let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001)))
I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever.
Can you help me to understand this better and to work around it ?
Regards Robert Larice
Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Thank You Daniel, I have exactly the same (ext:lisp-implementation-vcs-id) output. Can you send me the stdout of your (uncached/fresh) ./configure ? I would try to compare it with mine watching out for some difference. I did install these debian packages libgc-dev libffi-dev libatomic-ops-dev pkg-config libgmp-dev libx11-dev qtcreator qtbase5-dev qt5-default cl-quicklisp in addition to whatever I had before on my machine. Further, quicklisp gave me these versions in $HOME/quicklisp/dists/quicklisp/software alexandria-20170830-git bordeaux-threads-v0.8.6 lparallel-20160825-git Is there some difference to yours ? Best Regards, Robert Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
I can't reproduce the problem with the newest ECL code (from git develop branch) on my host. lparallel works fine and this snippet does terminate after timeout.
CL-USER> (ext:lisp-implementation-vcs-id) "ba6e6ddde780c097918673f16c7aba05f354d022"
Best regards, Daniel
W dniu nie, 02.09.2018 o godzinie 12∶27 +0200, użytkownik Robert Larice napisał:
I tried to understand where the issue is located.
In :lparallel file lparallel-20160825-git/src/thread-util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux-threads:condition-wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux-threads:condition-wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue:
(ql:quickload :bordeaux-threads)
(let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001)))
I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever.
Can you help me to understand this better and to work around it ?
Regards Robert Larice
Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Hello Daniel, I continued to search for the culprit. First I tried a second machine, which showed exactly the same problem. Thus I think you might not have used the same bordeaux thread version, I have bordeaux-threads-v0.8.6 The minimised snippet can be further minimised to this: (ql:quickload :bordeaux-threads) (let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (handler-case (bordeaux-threads:with-timeout (0.001) (mp:condition-variable-wait cvar lock)) (bordeaux-threads:timeout () nil)))) and still showes the same "hang" Then I started to macroexpand the stuff, and saw the with-timeout depends on interrupting a thread to terminate it after timeout. Merely guessing, I tried to wrap the whole code into a (mp:with-interrupts ... ) and voila, the piece starts to work properly. But actually, if I've glimpsed the doc correctly, the interrupts should have been enabled by default. Thus it seems either the bordeaux threads library is missing a with-interrupts, or there is a bug in ecl which doesn't enable these interrupts per default. Best Regards, Robert Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
I can't reproduce the problem with the newest ECL code (from git develop branch) on my host. lparallel works fine and this snippet does terminate after timeout.
CL-USER> (ext:lisp-implementation-vcs-id) "ba6e6ddde780c097918673f16c7aba05f354d022"
Best regards, Daniel
W dniu nie, 02.09.2018 o godzinie 12∶27 +0200, użytkownik Robert Larice napisał:
I tried to understand where the issue is located.
In :lparallel file lparallel-20160825-git/src/thread-util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux-threads:condition-wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux-threads:condition-wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue:
(ql:quickload :bordeaux-threads)
(let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001)))
I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever.
Can you help me to understand this better and to work around it ?
Regards Robert Larice
Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Hey Robert, nice analysis. While I have the newest ECL and the newest bordeaux- threads I've tried the snippet in SLIME REPL. CL-USER> si:*interrupts-enabled* T But when I run this snippet from console I have:
si:*interrupts-enabled* NIL
Moreover:
(bt:make-thread (lambda () (print `(jd ,si:*interrupts-enabled*)))) (JD T)
So it is a bug in ECL - interrupts are disabled indeed in the main thread by default. I'm looking into it at this very moment. Thank you for investigating. When fix is merged into develop branch I'll notify you. Best regards, Daniel W dniu pon, 03.09.2018 o godzinie 22∶31 +0200, użytkownik Robert Larice napisał:
Hello Daniel,
I continued to search for the culprit. First I tried a second machine, which showed exactly the same problem. Thus I think you might not have used the same bordeaux thread version, I have bordeaux-threads-v0.8.6
The minimised snippet can be further minimised to this: (ql:quickload :bordeaux-threads) (let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (handler-case (bordeaux-threads:with-timeout (0.001) (mp:condition-variable-wait cvar lock)) (bordeaux-threads:timeout () nil)))) and still showes the same "hang"
Then I started to macroexpand the stuff, and saw the with-timeout depends on interrupting a thread to terminate it after timeout. Merely guessing, I tried to wrap the whole code into a (mp:with-interrupts ... ) and voila, the piece starts to work properly. But actually, if I've glimpsed the doc correctly, the interrupts should have been enabled by default.
Thus it seems either the bordeaux threads library is missing a with-interrupts, or there is a bug in ecl which doesn't enable these interrupts per default.
Best Regards, Robert
Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
I can't reproduce the problem with the newest ECL code (from git develop branch) on my host. lparallel works fine and this snippet does terminate after timeout.
CL-USER> (ext:lisp-implementation-vcs-id) "ba6e6ddde780c097918673f16c7aba05f354d022"
Best regards, Daniel
W dniu nie, 02.09.2018 o godzinie 12∶27 +0200, użytkownik Robert Larice napisał:
I tried to understand where the issue is located.
In :lparallel file lparallel-20160825-git/src/thread-util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux-threads:condition- wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux-threads:condition- wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue:
(ql:quickload :bordeaux-threads)
(let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001)))
I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever.
Can you help me to understand this better and to work around it ?
Regards Robert Larice
Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Hello Daniel, if I grep for interrupts-enabled I see both si::*interrupts-enabled* and ext::*interrupts-enabled* and there is at least one or two commits which did rename such things. perhaps there is some confusion. with my debian dished SLIME I've both NIL, si::*interrupts-enabled* NIL ext::*interrupts-enabled* NIL Best Regards, Robert Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
nice analysis. While I have the newest ECL and the newest bordeaux- threads I've tried the snippet in SLIME REPL.
CL-USER> si:*interrupts-enabled* T
But when I run this snippet from console I have:
si:*interrupts-enabled* NIL
Moreover:
(bt:make-thread (lambda () (print `(jd ,si:*interrupts-enabled*)))) (JD T)
So it is a bug in ECL - interrupts are disabled indeed in the main thread by default. I'm looking into it at this very moment. Thank you for investigating. When fix is merged into develop branch I'll notify you.
Best regards, Daniel
W dniu pon, 03.09.2018 o godzinie 22∶31 +0200, użytkownik Robert Larice napisał:
Hello Daniel,
I continued to search for the culprit. First I tried a second machine, which showed exactly the same problem. Thus I think you might not have used the same bordeaux thread version, I have bordeaux-threads-v0.8.6
The minimised snippet can be further minimised to this: (ql:quickload :bordeaux-threads) (let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (handler-case (bordeaux-threads:with-timeout (0.001) (mp:condition-variable-wait cvar lock)) (bordeaux-threads:timeout () nil)))) and still showes the same "hang"
Then I started to macroexpand the stuff, and saw the with-timeout depends on interrupting a thread to terminate it after timeout. Merely guessing, I tried to wrap the whole code into a (mp:with-interrupts ... ) and voila, the piece starts to work properly. But actually, if I've glimpsed the doc correctly, the interrupts should have been enabled by default.
Thus it seems either the bordeaux threads library is missing a with-interrupts, or there is a bug in ecl which doesn't enable these interrupts per default.
Best Regards, Robert
Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
I can't reproduce the problem with the newest ECL code (from git develop branch) on my host. lparallel works fine and this snippet does terminate after timeout.
CL-USER> (ext:lisp-implementation-vcs-id) "ba6e6ddde780c097918673f16c7aba05f354d022"
Best regards, Daniel
W dniu nie, 02.09.2018 o godzinie 12∶27 +0200, użytkownik Robert Larice napisał:
I tried to understand where the issue is located.
In :lparallel file lparallel-20160825-git/src/thread-util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux-threads:condition- wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux-threads:condition- wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue:
(ql:quickload :bordeaux-threads)
(let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001)))
I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever.
Can you help me to understand this better and to work around it ?
Regards Robert Larice
Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
To load "lparallel": Load 1 ASDF system: lparallel ; Loading "lparallel"
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Hey Robert, I've found the issue and it is related to unwind-protect changes. Symbol is exported from both packages, but it is the same symbol. Problem doesn't manifest itself in this merge request[1]. Hopefully running ECL from this unmerged branch solves your problem too. Thank you for your persistance and investigation. Best regards, Daniel [1] https://gitlab.com/embeddable-common-lisp/ecl/merge_requests/119 W dniu wto, 04.09.2018 o godzinie 20∶08 +0200, użytkownik Robert Larice napisał:
Hello Daniel,
if I grep for interrupts-enabled I see both si::*interrupts-enabled* and ext::*interrupts-enabled* and there is at least one or two commits which did rename such things. perhaps there is some confusion.
with my debian dished SLIME I've both NIL, si::*interrupts-enabled* NIL ext::*interrupts-enabled* NIL
Best Regards, Robert
Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
nice analysis. While I have the newest ECL and the newest bordeaux- threads I've tried the snippet in SLIME REPL.
CL-USER> si:*interrupts-enabled* T
But when I run this snippet from console I have:
si:*interrupts-enabled*
NIL
Moreover:
(bt:make-thread (lambda () (print `(jd ,si:*interrupts- enabled*))))
(JD T)
So it is a bug in ECL - interrupts are disabled indeed in the main thread by default. I'm looking into it at this very moment. Thank you for investigating. When fix is merged into develop branch I'll notify you.
Best regards, Daniel
W dniu pon, 03.09.2018 o godzinie 22∶31 +0200, użytkownik Robert Larice napisał:
Hello Daniel,
I continued to search for the culprit. First I tried a second machine, which showed exactly the same problem. Thus I think you might not have used the same bordeaux thread version, I have bordeaux-threads-v0.8.6
The minimised snippet can be further minimised to this: (ql:quickload :bordeaux-threads) (let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (handler-case (bordeaux-threads:with-timeout (0.001) (mp:condition-variable-wait cvar lock)) (bordeaux-threads:timeout () nil)))) and still showes the same "hang"
Then I started to macroexpand the stuff, and saw the with-timeout depends on interrupting a thread to terminate it after timeout. Merely guessing, I tried to wrap the whole code into a (mp:with-interrupts ... ) and voila, the piece starts to work properly. But actually, if I've glimpsed the doc correctly, the interrupts should have been enabled by default.
Thus it seems either the bordeaux threads library is missing a with-interrupts, or there is a bug in ecl which doesn't enable these interrupts per default.
Best Regards, Robert
Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
I can't reproduce the problem with the newest ECL code (from git develop branch) on my host. lparallel works fine and this snippet does terminate after timeout.
CL-USER> (ext:lisp-implementation-vcs-id) "ba6e6ddde780c097918673f16c7aba05f354d022"
Best regards, Daniel
W dniu nie, 02.09.2018 o godzinie 12∶27 +0200, użytkownik Robert Larice napisał:
I tried to understand where the issue is located.
In :lparallel file lparallel-20160825-git/src/thread- util.lisp there is a stanza which tries to check for the implementation of the :timeout capability
;;; Check for timeout parameter in bordeaux- threads:condition- wait. (eval-when (:compile-toplevel :execute) ;; use special to defeat compiler analysis (defparameter *condition-wait* #'bordeaux- threads:condition- wait)
(flet ((has-condition-wait-timeout-p () ...
I tried to minimise this to a small standalone piece which can be used to examine the issue:
(ql:quickload :bordeaux-threads)
(let ((lock (bordeaux-threads:make-lock)) (cvar (bordeaux-threads:make-condition-variable))) (bordeaux-threads:with-lock-held (lock) (bordeaux-threads:condition-wait cvar lock :timeout 0.001)))
I think the :timeout doesn't seem to work properly and thus the condition-wait waits forever.
Can you help me to understand this better and to work around it ?
Regards Robert Larice
Robert Larice <Robert.Larice@t-online.de> writes:
Hello,
can you provide me some hints for the following problem ?
I've compiled ecl from git on a debian stretch machine. Then I've tried to compile the "qt" example, which did hang when loading "lparallel" So I started "ecl" from a shell, and did execute (ql:quickload :lparallel) which presents me: > To load "lparallel": > Load 1 ASDF system: > lparallel > ; Loading "lparallel" then the process fell silent. "top" doesn't show "ecl" to consume cpu cycles. Its waiting for something and doesn't proceed.
Regards, Robert Larice
Thank You, I've fetched the branch with your fix and recompiled. The issue is gone. Linking the qt example did fail. The following diff did fix this too. modified examples/ecl_qt/build_static.lisp @@ -3,6 +3,7 @@ (asdf:make-build :lisp-envi :type :static-library - :move-here "qt/") + :move-here "qt/" + :init-name "init_lib_LISP_ENVI") Best Regards, Robert Daniel Kochmański <daniel@turtleware.eu> writes:
Hey Robert,
I've found the issue and it is related to unwind-protect changes. Symbol is exported from both packages, but it is the same symbol. Problem doesn't manifest itself in this merge request[1]. Hopefully running ECL from this unmerged branch solves your problem too. Thank you for your persistance and investigation.
Best regards, Daniel
[1] https://gitlab.com/embeddable-common-lisp/ecl/merge_requests/119
participants (2)
-
Daniel Kochmański -
Robert Larice