Hi,
I'm not 100% sure this is a SLIME question, an ASDF question, or a Common Lisp question, but I'll ask it here anyway.
In my program I'd like to have *read-default-float-format* be long-float instead of the default single-float. To accomplish this, I added (eval-when (:compile-toplevel :load-toplevel :execute) (setq *read-default-float-format* 'long-float)) to package.lisp, which should be the first file to be compiled/loaded by ASDF.
Now if I do a ,load-system in the SLIME REPL, everything is fine: *read-default-float-format* evaluates to long-float during compilation and also does in the REPL. However, if I open one of my files, and add (defun foo () 50.0), do a C-M-x to compile it, and then call (foo) in the REPL, I get 50.0f0 instead of 50.0d0. C-c C-k doesn't help either, and C-x C-e on the 50.0 also produces a single-float. If I then save the file and do another ,load-system, then (foo) in the REPL /will/ return 50.0d0, as expected.
How can I get C-M-x, C-c C-k and C-x C-e to use long-floats too? Am I doing something wrong that causes them to behave differently from ,load-system? I'm using SLIME and Emacs from CVS, and have set inferior-lisp-program and slime-multiprocessing to "sbcl" and t, respectively.
Kind regards,
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
Hi,
I'm not 100% sure this is a SLIME question, an ASDF question, or a Common Lisp question, but I'll ask it here anyway.
In my program I'd like to have *read-default-float-format* be long-float instead of the default single-float. To accomplish this, I added (eval-when (:compile-toplevel :load-toplevel :execute) (setq *read-default-float-format* 'long-float)) to package.lisp, which should be the first file to be compiled/loaded by ASDF.
Now if I do a ,load-system in the SLIME REPL, everything is fine: *read-default-float-format* evaluates to long-float during compilation and also does in the REPL. However, if I open one of my files, and add (defun foo () 50.0), do a C-M-x to compile it, and then call (foo) in the REPL, I get 50.0f0 instead of 50.0d0. C-c C-k doesn't help either, and C-x C-e on the 50.0 also produces a single-float. If I then save the file and do another ,load-system, then (foo) in the REPL /will/ return 50.0d0, as expected.
How can I get C-M-x, C-c C-k and C-x C-e to use long-floats too? Am I doing something wrong that causes them to behave differently from ,load-system? I'm using SLIME and Emacs from CVS, and have set inferior-lisp-program and slime-multiprocessing to "sbcl" and t, respectively.
Kind regards,
Dirk Gerrits
Anyone? This is really annoying, as it makes C-M-x, C-c C-k and C-x C-e completely unusable for this project. (Well unless there just happens to be no floating point literal in the evaluated code.)
Thanks in advance,
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
(eval-when (:compile-toplevel :load-toplevel :execute) (setq *read-default-float-format* 'long-float))
[...]
Anyone? This is really annoying, as it makes C-M-x, C-c C-k and C-x C-e completely unusable for this project. (Well unless there just happens to be no floating point literal in the evaluated code.)
This is probably specific to the Lisp implementation. Which are you using?
If you're using ACL then you should probably use excl:setq-default instead of setq. Otherwise new threads will use the default instead of what you want.
See http://www.franz.com/support/documentation/6.2/doc/pages/operators/top-level...
Luke Gorrie luke@synap.se writes:
Dirk Gerrits dirk@dirkgerrits.com writes:
(eval-when (:compile-toplevel :load-toplevel :execute) (setq *read-default-float-format* 'long-float))
[...]
Anyone? This is really annoying, as it makes C-M-x, C-c C-k and C-x C-e completely unusable for this project. (Well unless there just happens to be no floating point literal in the evaluated code.)
This is probably specific to the Lisp implementation. Which are you using?
If you're using ACL then you should probably use excl:setq-default instead of setq. Otherwise new threads will use the default instead of what you want.
See http://www.franz.com/support/documentation/6.2/doc/pages/operators/top-level...
Sounds plausible. As I said in my original post however, I'm using SBCL with multithreading. I've been looking in SBCL the manual, but I can't find anything similar to setq-default. As far as I know, setq is the correct thing to use for new threads to inherit the value for special variables.
Dirk Gerrits
Luke Gorrie luke@synap.se writes:
Dirk Gerrits dirk@dirkgerrits.com writes:
(eval-when (:compile-toplevel :load-toplevel :execute) (setq *read-default-float-format* 'long-float))
[...]
Anyone? This is really annoying, as it makes C-M-x, C-c C-k and C-x C-e completely unusable for this project. (Well unless there just happens to be no floating point literal in the evaluated code.)
This is probably specific to the Lisp implementation. Which are you using?
If you're using ACL then you should probably use excl:setq-default instead of setq. Otherwise new threads will use the default instead of what you want.
See http://www.franz.com/support/documentation/6.2/doc/pages/operators/top-level...
Sounds plausible. As I said in my original post however, I'm using SBCL with multithreading. I've been looking in SBCL the manual, but I can't find anything similar to setq-default. As far as I know, setq is the correct thing to use for new child threads to inherit the values of special variables. So apparently my setq is not being executed in the "root thread"?
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
Luke Gorrie luke@synap.se writes:
Dirk Gerrits dirk@dirkgerrits.com writes:
(eval-when (:compile-toplevel :load-toplevel :execute) (setq *read-default-float-format* 'long-float))
[...]
Anyone? This is really annoying, as it makes C-M-x, C-c C-k and C-x C-e completely unusable for this project. (Well unless there just happens to be no floating point literal in the evaluated code.)
This is probably specific to the Lisp implementation. Which are you using?
If you're using ACL then you should probably use excl:setq-default instead of setq. Otherwise new threads will use the default instead of what you want.
See http://www.franz.com/support/documentation/6.2/doc/pages/operators/top-level...
Sounds plausible. As I said in my original post however, I'm using SBCL with multithreading. I've been looking in SBCL the manual, but I can't find anything similar to setq-default. As far as I know, setq is the correct thing to use for new child threads to inherit the values of special variables. So apparently my setq is not being executed in the "root thread"?
Okay I still haven't found a solution, so maybe I should just type "l0"'s all the time and leave *read-default-float-format* at its default setting?
Kind regards, and a happy new year,
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
Okay I still haven't found a solution, so maybe I should just type "l0"'s all the time and leave *read-default-float-format* at its default setting?
SBCL inherits values of special variables from the parent thread; I've slightly lost track of how slime works multithreadedly -- does it spawn a new thread for each evaluation, or does it farm them off to a worker thread?
Also, each thread has its own binding of the various CL specials, so setting the value in one thread won't affect the values in other threads.
Given your description, I would suggest trying to set *read-default-float-format* in the slime thread which does some work: try first setting it in the *inferior-lisp* buffer. If that's not the right thread to set it in, and you can find the right one but can't set the variable in it, shout and I'll make a (setf (symbol-value-in-thread ...) ...) function available.
Cheers,
Christophe
Christophe Rhodes csr21@cam.ac.uk writes:
I've slightly lost track of how slime works multithreadedly -- does it spawn a new thread for each evaluation, or does it farm them off to a worker thread?
I'm not familiar with SLIME's internals, but I've done some tests with SB-THREAD:CURRENT-THREAD-ID. In my package.lisp file, I've got the following:
(eval-when (:compile-toplevel :load-toplevel :execute) (with-open-file (stream "thread-id.txt" :direction :output :if-exists :append) (print (sb-thread:current-thread-id) stream))
;; Use long-floats instead of single-floats by default. (setq *read-default-float-format* 'long-float))
When I do a ,load-system of my ASDF system from the REPL, the processing of this file generated thread-ids:
23673 23710 23710
The REPL has thread-id 23710 as well. So I guess COMPILE-OP is done in a seperate thread but LOAD-OP is done in the REPL thread? Using the *inferior-lisp* buffer's REPL reports 23706, by the way.
Putting the point after (sb-thread:current-thread-id) in the package.lisp file and then pressing C-x C-e produces a different thread-id each time I do it: 23761, 23762, 23763, ... Similarly for C-M-x.
So it seems that these commands spawn a new thread each time. These threads don't seem to be children of the REPL's thread, since *read-default-float-format* evaluates to LONG-FLOAT in the REPL, but doing a C-x C-e on *read-default-float-format* in package.lisp yields SINGLE-FLOAT.
When I do a (setq *read-default-float-format* 'long-float) in the *inferior-lisp* buffer, the above situation doesn't change, so the spawned threads don't seem to be children of *inferior-lisp*'s thread either.
Is there perhaps any way in SBCL for me to examine the parent/child relations between threads in order to give you more information?
Given your description, I would suggest trying to set *read-default-float-format* in the slime thread which does some work: try first setting it in the *inferior-lisp* buffer. If that's not the right thread to set it in, and you can find the right one but can't set the variable in it, shout and I'll make a (setf (symbol-value-in-thread ...) ...) function available.
Something like this?
#+:sb-thread (when (find-package :swank-loader) (setf (sb-thread::symbol-value-in-thread *read-default-float-format* (the-right-thread)) 'long-float))
Sounds reasonable, but I haven't found the right thread yet. Any SLIME developers who can help?
Thanks for the help so far.
Kind regards,
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
Something like this?
#+:sb-thread (when (find-package :swank-loader) (setf (sb-thread::symbol-value-in-thread *read-default-float-format* (the-right-thread)) 'long-float))
Sounds reasonable, but I haven't found the right thread yet. Any SLIME developers who can help?
The thread tree looks like this:
inferior-lisp (== SBCL REPL) _ socket server [usually dead] _ reader thread _ repl thread _ control thread _ worker thread1 _ worker thread2 ...
REPL command are executed by the repl thread. For the other request, a new worker thread is created by the control thread. So, the-right-thread(s) are probably the control thread and the repl thread.
sb-thread::symbol-value-in-thread doesn't appear to be setf-able here, but
(sb-thread:interrupt-thread (swank::connection.control-thread swank::*emacs-connection*) (lambda () (setq *read-default-float-format* 'long-float)))
seems to work.
The issue came up before but I still don't know what the proper fix is. SBCL is the only Lisp with the inherit-dynamic-variables-in-new-threads semantics. Implementing non-inheriting seems to be difficult in SBCL and implementing inheriting is difficult in the other Lisps. We could probably offer something like swank:setq-default to set initial values in the worker threads. Other ideas?
Helmut.
Helmut Eller e9626484@stud3.tuwien.ac.at writes:
Dirk Gerrits dirk@dirkgerrits.com writes:
Something like this?
#+:sb-thread (when (find-package :swank-loader) (setf (sb-thread::symbol-value-in-thread *read-default-float-format* (the-right-thread)) 'long-float))
Sounds reasonable, but I haven't found the right thread yet. Any SLIME developers who can help?
The thread tree looks like this:
inferior-lisp (== SBCL REPL) _ socket server [usually dead] _ reader thread _ repl thread _ control thread _ worker thread1 _ worker thread2 ...
REPL command are executed by the repl thread. For the other request, a new worker thread is created by the control thread. So, the-right-thread(s) are probably the control thread and the repl thread.
This seems strange. Has any of this changed recently? It doesn't seem to explain the behaviour I reported earlier:
In my package.lisp file, I've got the following:
(eval-when (:compile-toplevel :load-toplevel :execute) (with-open-file (stream "thread-id.txt" :direction :output :if-exists :append) (print (sb-thread:current-thread-id) stream))
;; Use long-floats instead of single-floats by default. (setq *read-default-float-format* 'long-float))
When I do a ,load-system of my ASDF system from the REPL, the processing of this file generated thread-ids:
23673 23710 23710
The REPL has thread-id 23710 as well. So I guess COMPILE-OP is done in a seperate thread but LOAD-OP is done in the REPL thread? Using the *inferior-lisp* buffer's REPL reports 23706, by the way.
This says inferior-lisp /= SBCL REPL, and the SETQ was apparently executed in the REPL thread yet the binding wasn't inherited by the worker threads.
I'll try this out again soon. If the thread tree is really as you describe, and ,load-system is done in the SCBL REPL thread, then there shouldn't be a problem at all, right?
sb-thread::symbol-value-in-thread doesn't appear to be setf-able here,
Indeed, Christophe asked whether it would help me if he added it to SBCL.
but
(sb-thread:interrupt-thread (swank::connection.control-thread swank::*emacs-connection*) (lambda () (setq *read-default-float-format* 'long-float)))
seems to work.
Okay, I'll try this.
The issue came up before but I still don't know what the proper fix is. SBCL is the only Lisp with the inherit-dynamic-variables-in-new-threads semantics.
I thought AllegroCL 7 and Corman Common Lisp (and possibly others) also behaved in this way? In any case, it seems to me that this is the reasonable thing to do with special variable bindings in a multithreaded setting.
Implementing non-inheriting seems to be difficult in SBCL and implementing inheriting is difficult in the other Lisps. We could probably offer something like swank:setq-default to set initial values in the worker threads. Other ideas?
I reckon the problem is a lot deeper than SBCL + SLIME, but SWANK:SETQ-DEFAULT sounds like a good workaround.
Kind regards,
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
REPL command are executed by the repl thread. For the other request, a new worker thread is created by the control thread. So, the-right-thread(s) are probably the control thread and the repl thread.
This seems strange. Has any of this changed recently? It doesn't seem to explain the behaviour I reported earlier:
No; not since June.
In my package.lisp file, I've got the following:
(eval-when (:compile-toplevel :load-toplevel :execute) (with-open-file (stream "thread-id.txt" :direction :output :if-exists :append) (print (sb-thread:current-thread-id) stream))
;; Use long-floats instead of single-floats by default. (setq *read-default-float-format* 'long-float))When I do a ,load-system of my ASDF system from the REPL, the processing of this file generated thread-ids:
23673 23710 23710
The REPL has thread-id 23710 as well. So I guess COMPILE-OP is done in a seperate thread but LOAD-OP is done in the REPL thread? Using the *inferior-lisp* buffer's REPL reports 23706, by the way.
This says inferior-lisp /= SBCL REPL, and the SETQ was apparently executed in the REPL thread yet the binding wasn't inherited by the worker threads.
I think only the last 2 lines are created by one ,load-op. At least (load (compile-file ...)) produces only 2 lines. It would also be strange to have a thread id (23673) smaller than the id of the SBCL REPL thread (23706).
I'll try this out again soon. If the thread tree is really as you describe, and ,load-system is done in the SCBL REPL thread, then there shouldn't be a problem at all, right?
Only if you do it before the other threads are created, of course. If you do it afterwards, the value in the other threads remains unchanged.
The issue came up before but I still don't know what the proper fix is. SBCL is the only Lisp with the inherit-dynamic-variables-in-new-threads semantics.
I thought AllegroCL 7 and Corman Common Lisp (and possibly others) also behaved in this way? In any case, it seems to me that this is the reasonable thing to do with special variable bindings in a multithreaded setting.
Not sure about Corman, but in Allegro the dynamic environment in new thread is initially empty. See http://www.franz.com/support/documentation/7.0/doc/multiprocessing.htm#dynam...
Implementing non-inheriting seems to be difficult in SBCL and implementing inheriting is difficult in the other Lisps. We could probably offer something like swank:setq-default to set initial values in the worker threads. Other ideas?
I reckon the problem is a lot deeper than SBCL + SLIME, but SWANK:SETQ-DEFAULT sounds like a good workaround.
I added a new variable *default-worker-thread-bindings* instead. The intended use is something like
(push (cons '*read-default-float-format* 'long-float) *default-worker-thread-bindings*)
Only new worker threads are affected by this variable. To change the variable in the REPL you have to evaluate a setq in the REPL thread.
Helmut.
Helmut Eller e9626484@stud3.tuwien.ac.at writes:
Dirk Gerrits dirk@dirkgerrits.com writes:
I'll try this out again soon. If the thread tree is really as you describe, and ,load-system is done in the SCBL REPL thread, then there shouldn't be a problem at all, right?
Only if you do it before the other threads are created, of course. If you do it afterwards, the value in the other threads remains unchanged.
Okay, now I'm confused.
As I understood it, special variable /bindings/ are inherited by child threads in SBCL. Doing a SETQ in any of those threads should affect all those threads unless they have created a new, local binding for that variable with LET.
Are you saying that what I'm seeing is because SBCL threads inherit special variable /values/ inside new, local /bindings/? With 47 special variables in the COMMON-LISP package alone, that can't be right, so I must have misunderstood you.
Not sure about Corman, but in Allegro the dynamic environment in new thread is initially empty. See http://www.franz.com/support/documentation/7.0/doc/multiprocessing.htm#dynam...
This describes exactly what I thought SBCL's (and Corman's) semantics to be. Can you explain the difference with http://www.sbcl.org/manual/Special-Variables.html ?
I added a new variable *default-worker-thread-bindings* instead. The intended use is something like
(push (cons '*read-default-float-format* 'long-float) *default-worker-thread-bindings*)
Only new worker threads are affected by this variable. To change the variable in the REPL you have to evaluate a setq in the REPL thread.
This seems to work like a charm. Thanks a lot!
Kind regards,
Dirk Gerrits
Dirk Gerrits dirk@dirkgerrits.com writes:
Okay, now I'm confused.
As I understood it, special variable /bindings/ are inherited by child threads in SBCL. Doing a SETQ in any of those threads should affect all those threads unless they have created a new, local binding for that variable with LET.
Are you saying that what I'm seeing is because SBCL threads inherit special variable /values/ inside new, local /bindings/? With 47 special variables in the COMMON-LISP package alone, that can't be right, so I must have misunderstood you.
Yes, it's closer to the latter version. "inheriting the binding" was probably not the correct term. Sorry for the confusion.
If the parent thread has a dynamic binding the child will also have a dynamic binding and the value will be inherited from the parent. The dynamic binding itself is private to a thread and SETQ-ing it will not change the value in other threads. Only the global (non-dynamic) binding is shared.
Creating a thread is probably not very fast, but Common Lisp was not designed with concurrency in mind.
Not sure about Corman, but in Allegro the dynamic environment in new thread is initially empty. See http://www.franz.com/support/documentation/7.0/doc/multiprocessing.htm#dynam...
This describes exactly what I thought SBCL's (and Corman's) semantics to be. Can you explain the difference with http://www.sbcl.org/manual/Special-Variables.html ?
Here's an example:
(defvar *dvar* 10)
(let ((*dvar* 20) (lock t)) (sb-thread:make-thread (lambda () (print *dvar*) (setq *dvar* 30) (setq lock nil))) (loop while lock do (sleep 0.2)) (print *dvar*))
This prints
20 20
in SBCL. The equivalent (MAKE-THREAD replaced with PROCESS-RUN-FUNCTION) in Allegro prints:
10 20
Helmut.
Helmut Eller e9626484@stud3.tuwien.ac.at writes:
If the parent thread has a dynamic binding the child will also have a dynamic binding and the value will be inherited from the parent. The dynamic binding itself is private to a thread and SETQ-ing it will not change the value in other threads. Only the global (non-dynamic) binding is shared.
What the ... ?!?!
Creating a thread is probably not very fast, but Common Lisp was not designed with concurrency in mind.
Well I assume most special variables will be "at the global binding", but still...
Here's an example:
(defvar *dvar* 10)
(let ((*dvar* 20) (lock t)) (sb-thread:make-thread (lambda () (print *dvar*) (setq *dvar* 30) (setq lock nil))) (loop while lock do (sleep 0.2)) (print *dvar*))
This prints
20 20
in SBCL. The equivalent (MAKE-THREAD replaced with PROCESS-RUN-FUNCTION) in Allegro prints:
10 20
You're absolutely right. And leaving out the (*dvar* 20) from the LET prints 10 30 for both Lisps.
Quite frankly, I find SBCL's behaviour baffling. Let's see what replies I get to my request for a rationale on sbcl-devel.
Kind regards,
Dirk Gerrits
-
Christophe Rhodes -
Dirk Gerrits -
Helmut Eller -
Luke Gorrie