[parenscript-devel] A simpler way to do multiple value return? (Was: PS test failures in 7be9b45)
Those test failures make sense now - thanks. Also, this comment reminded me of something: < the callee.caller property for functions, which multiple value return depends on > I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations. A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable? Say this variable is called *spillover*. Then this: (defun blah () (values 1 2 3)) (defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c))) ...might compile to: function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; }; There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are. Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value attached to callee and restore it using a try/finally at the end. Second, it would fix a known problem. PS breaks when you have an interloper like FOO here: (defun blah () (values 1 2 3)) (defun foo () (blah)) (defun bar () (multiple-value-bind (a b c) (foo) (+ a b c))) BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6. Could we get away with this, or what am I missing? Daniel On Tue, Aug 28, 2012 at 6:00 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
Hi Daniel,
Yes, those two failures in the eval test suite are expected. CL-JavaScript doesn't have the callee.caller property for functions, which multiple value return depends on. I wasn't sure where to comment those tests out, so I left them in to remind myself to add callee.caller to CL-JavaScript (I've already talked to Marijn Haverbeke about that).
Thank you, Vladimir
On Mon, Aug 27, 2012 at 11:58 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
I've rebased my PS LOOP extensions [1] onto the latest commit (7be9b45) and recompiled Skysheet. The generated JS looks fine. There was one glitch that I'll report separately along with a workaround. Before pushing the LOOP extensions onto master, though, I want to update any relevant PS tests. Some will fail because the LOOP output has changed quite a bit. Unfortunately I'm also seeing failures when I run the tests in 7be9b45, which is prior to any of these LOOP changes. I've pasted the output below [2]. It doesn't look like these failures are related to work in ps-loop.lisp, so I'll just ignore them for the time being, but Vladimir can you please comment on whether you know about them or whether there's something unexpected going on?
Daniel
[1] These are the constructs FOR..OF and MAP..TO, plus a change to FOR..ON, that I described in my email to this list on April 11. They are currently sitting in the "loop" branch. Rebasing them was nontrivial because of Boris' additions to ps-loop.lisp, but it seems to have all gone ok. Boris, if you're reading this, please look out for any regressions once I push these changes and let us know if you notice anything.
[2] Running output tests:
........................................................................................................................................................................................................................................................................................................................................................................................................................................
Did 424 checks. Pass: 424 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running package system tests: ......... Did 9 checks. Pass: 9 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running CL-JavaScript eval tests: ...........................f...............X...................... Did 66 checks. Pass: 64 (96%) Skip: 0 ( 0%) Fail: 2 ( 3%) Failure Details: -------------------------------- mv-return1 []: Unexpected Error: #<cl-js:js-condition #x30200155257D> [js] TypeError: undefined has no properties... -------------------------------- -------------------------------- dynamic-extent-function-return-values []: (funcall (if (typep #:g36204 'structure-object) #'equalp #'equal) #:g36204 (jsarray '(1 2 3))) was NIL.. --------------------------------
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Those test failures make sense now - thanks. Also, this comment reminded me of something: < the callee.caller property for functions, which multiple value return depends on > I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations. A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable? Say this variable is called *spillover*. Then this: (defun blah () (values 1 2 3)) (defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c))) ...might compile to: function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; }; There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are. Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value and restore it using a try/finally at the end. Second, it would fix a known problem. PS breaks when you introduce an interloper like FOO here: (defun blah () (values 1 2 3)) (defun foo () (blah)) (defun bar () (multiple-value-bind (a b c) (foo) (+ a b c))) BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6. Could we get away with this, or what am I missing? Daniel
The counter-example to using a global variable as I remember it from the last discussion was this case: (defun blah () (values 1 2 3)) (defun foo () (blah) (some-random-js-function)) (defun bar () (multiple-value-bind (a b c) (foo) (+ a b c))) Now a call to bar returns 6, when it shouldn't. I think it might be possible to use another global variable as a flag that's set and checked by multiple-value aware PS functions, but I need to think about how to make it work. Vladimir On Tue, Aug 28, 2012 at 10:07 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value attached to callee and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you have an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
On Tue, Aug 28, 2012 at 6:00 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
Hi Daniel,
Yes, those two failures in the eval test suite are expected. CL-JavaScript doesn't have the callee.caller property for functions, which multiple value return depends on. I wasn't sure where to comment those tests out, so I left them in to remind myself to add callee.caller to CL-JavaScript (I've already talked to Marijn Haverbeke about that).
Thank you, Vladimir
On Mon, Aug 27, 2012 at 11:58 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
I've rebased my PS LOOP extensions [1] onto the latest commit (7be9b45) and recompiled Skysheet. The generated JS looks fine. There was one glitch that I'll report separately along with a workaround. Before pushing the LOOP extensions onto master, though, I want to update any relevant PS tests. Some will fail because the LOOP output has changed quite a bit. Unfortunately I'm also seeing failures when I run the tests in 7be9b45, which is prior to any of these LOOP changes. I've pasted the output below [2]. It doesn't look like these failures are related to work in ps-loop.lisp, so I'll just ignore them for the time being, but Vladimir can you please comment on whether you know about them or whether there's something unexpected going on?
Daniel
[1] These are the constructs FOR..OF and MAP..TO, plus a change to FOR..ON, that I described in my email to this list on April 11. They are currently sitting in the "loop" branch. Rebasing them was nontrivial because of Boris' additions to ps-loop.lisp, but it seems to have all gone ok. Boris, if you're reading this, please look out for any regressions once I push these changes and let us know if you notice anything.
[2] Running output tests:
........................................................................................................................................................................................................................................................................................................................................................................................................................................ Did 424 checks. Pass: 424 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running package system tests: ......... Did 9 checks. Pass: 9 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running CL-JavaScript eval tests: ...........................f...............X...................... Did 66 checks. Pass: 64 (96%) Skip: 0 ( 0%) Fail: 2 ( 3%) Failure Details: -------------------------------- mv-return1 []: Unexpected Error: #<cl-js:js-condition #x30200155257D> [js] TypeError: undefined has no properties... -------------------------------- -------------------------------- dynamic-extent-function-return-values []: (funcall (if (typep #:g36204 'structure-object) #'equalp #'equal) #:g36204 (jsarray '(1 2 3))) was NIL.. --------------------------------
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you introduce an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Good memory! This arguably moves the balance a bit in *favor* of the simpler implementation, because this bug is a little less likely to come up in practice than the existing bug PS. (Obviously it would be better to eliminate both bugs.) Is the compiler allowed to keep track of which functions return multiple values? If so, it could recognize that the call to BLAH is not a return value of FOO and compile FOO like this: function foo() { blah(); SPILLOVER = null; return someRandomJsFunction(); }; In other words, use it (where returning counts as a use) or lose it. Daniel On Tue, Aug 28, 2012 at 8:20 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
The counter-example to using a global variable as I remember it from the last discussion was this case:
(defun blah () (values 1 2 3))
(defun foo () (blah) (some-random-js-function))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
Now a call to bar returns 6, when it shouldn't. I think it might be possible to use another global variable as a flag that's set and checked by multiple-value aware PS functions, but I need to think about how to make it work.
Vladimir
On Tue, Aug 28, 2012 at 10:07 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value attached to callee and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you have an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
On Tue, Aug 28, 2012 at 6:00 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
Hi Daniel,
Yes, those two failures in the eval test suite are expected. CL-JavaScript doesn't have the callee.caller property for functions, which multiple value return depends on. I wasn't sure where to comment those tests out, so I left them in to remind myself to add callee.caller to CL-JavaScript (I've already talked to Marijn Haverbeke about that).
Thank you, Vladimir
On Mon, Aug 27, 2012 at 11:58 PM, Daniel Gackle <danielgackle@gmail.com
wrote:
I've rebased my PS LOOP extensions [1] onto the latest commit (7be9b45) and recompiled Skysheet. The generated JS looks fine. There was one glitch that I'll report separately along with a workaround. Before pushing the LOOP extensions onto master, though, I want to update any relevant PS tests. Some will fail because the LOOP output has changed quite a bit. Unfortunately I'm also seeing failures when I run the tests in 7be9b45, which is prior to any of these LOOP changes. I've pasted the output below [2]. It doesn't look like these failures are related to work in ps-loop.lisp, so I'll just ignore them for the time being, but Vladimir can you please comment on whether you know about them or whether there's something unexpected going on?
Daniel
[1] These are the constructs FOR..OF and MAP..TO, plus a change to FOR..ON, that I described in my email to this list on April 11. They are currently sitting in the "loop" branch. Rebasing them was nontrivial because of Boris' additions to ps-loop.lisp, but it seems to have all gone ok. Boris, if you're reading this, please look out for any regressions once I push these changes and let us know if you notice anything.
[2] Running output tests:
........................................................................................................................................................................................................................................................................................................................................................................................................................................
Did 424 checks. Pass: 424 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running package system tests: ......... Did 9 checks. Pass: 9 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running CL-JavaScript eval tests: ...........................f...............X...................... Did 66 checks. Pass: 64 (96%) Skip: 0 ( 0%) Fail: 2 ( 3%) Failure Details: -------------------------------- mv-return1 []: Unexpected Error: #<cl-js:js-condition #x30200155257D> [js] TypeError: undefined has no properties... -------------------------------- -------------------------------- dynamic-extent-function-return-values []: (funcall (if (typep #:g36204 'structure-object) #'equalp #'equal) #:g36204 (jsarray '(1 2 3))) was NIL.. --------------------------------
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you introduce an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
On Tue, Aug 28, 2012 at 7:20 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
The counter-example to using a global variable as I remember it from the last discussion was this case:
The last discussion: http://lists.common-lisp.net/pipermail/parenscript-devel/2009-October/000639...
(defun blah () (values 1 2 3))
(defun foo () (blah) (some-random-js-function))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
Now a call to bar returns 6, when it shouldn't. I think it might be possible to use another global variable as a flag that's set and checked by multiple-value aware PS functions, but I need to think about how to make it work.
Vladimir
On Tue, Aug 28, 2012 at 10:07 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
My guess at why your proposal won't work: It ignores knowledge of the call stack, which is practically necessary. Imagine a form (multiple-values-bind (a b c ...) (FOO ...)). Any solution of this with global variables will take the form: step 1: Do some stuff with global variable manipulation step 2: Call FOO step 3: Do some stuff to clean up. Pretty much any code can execute inside FOO. This includes a few scenarios: Scenario 1: A simple JS function, native or not, that doesn't call anything that returns multiple values. Your solution works. Scenario 2: A Parenscript function, written to manipulate your global variables properly. Your solution works. Scenario 3: A non-Parenscript function that calls a mv-returning Parenscript function. Can your solution handle Scenario 3? My guess is it cannot because it ignores the call stack. Just my quick guess. - Red
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value attached to callee and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you have an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
On Tue, Aug 28, 2012 at 6:00 PM, Vladimir Sedach <vsedach@gmail.com>
wrote:
Hi Daniel,
Yes, those two failures in the eval test suite are expected. CL-JavaScript doesn't have the callee.caller property for functions, which multiple value return depends on. I wasn't sure where to comment those tests out, so I left them in to remind myself to add callee.caller to CL-JavaScript (I've already talked to Marijn Haverbeke about that).
Thank you, Vladimir
On Mon, Aug 27, 2012 at 11:58 PM, Daniel Gackle <danielgackle@gmail.com
wrote:
I've rebased my PS LOOP extensions [1] onto the latest commit (7be9b45) and recompiled Skysheet. The generated JS looks fine. There was one glitch that I'll report separately along with a workaround. Before pushing the LOOP extensions onto master, though, I want to update any relevant PS tests. Some will fail because the LOOP output has changed quite a bit. Unfortunately I'm also seeing failures when I run the tests in 7be9b45, which is prior to any of these LOOP changes. I've pasted the output below [2]. It doesn't look like these failures are related to work in ps-loop.lisp, so I'll just ignore them for the time being, but Vladimir can you please comment on whether you know about them or whether there's something unexpected going on?
Daniel
[1] These are the constructs FOR..OF and MAP..TO, plus a change to FOR..ON, that I described in my email to this list on April 11. They are currently sitting in the "loop" branch. Rebasing them was nontrivial because of Boris' additions to ps-loop.lisp, but it seems to have all gone ok. Boris, if you're reading this, please look out for any regressions once I push these changes and let us know if you notice anything.
[2] Running output tests:
........................................................................................................................................................................................................................................................................................................................................................................................................................................
Did 424 checks. Pass: 424 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running package system tests: ......... Did 9 checks. Pass: 9 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running CL-JavaScript eval tests: ...........................f...............X...................... Did 66 checks. Pass: 64 (96%) Skip: 0 ( 0%) Fail: 2 ( 3%) Failure Details: -------------------------------- mv-return1 []: Unexpected Error: #<cl-js:js-condition #x30200155257D> [js] TypeError: undefined has no properties... -------------------------------- -------------------------------- dynamic-extent-function-return-values []: (funcall (if (typep #:g36204 'structure-object) #'equalp #'equal) #:g36204 (jsarray '(1 2 3))) was NIL.. --------------------------------
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you introduce an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Hi Red, I was hoping you'd chime in. I'll see your scenario 3 and raise you a 3a and a 3b: Scenario 3a: A non-Parenscript function that calls a mv-returning Parenscript function but only needs its first return value; Scenario 3b: A non-Parenscript function that calls a mv-returning Parenscript function and needs all its return values. 3a works fine as long as the MV implementation is careful to use a normal JS return to pass the first return value back to the caller. That's true both of what PS does today and of the global-var proposal. As for 3b (the scenario, not the hacker!), seems to me this can't work at all and there's no need to support it. If you're a non-PS function then by definition you can't use PS's MULTIPLE-VALUE-BIND to access the additional return values because the MV construct only exists in PS. I suppose if you really wanted to you could manually write JS to do whatever PS does to supply those values to the caller, but then you're not really a non-PS function anymore, so much as a manually-compiled PS function. Daniel On Tue, Aug 28, 2012 at 8:46 PM, Red Daly <reddaly@gmail.com> wrote:
On Tue, Aug 28, 2012 at 7:20 PM, Vladimir Sedach <vsedach@gmail.com>wrote:
The counter-example to using a global variable as I remember it from the last discussion was this case:
The last discussion: http://lists.common-lisp.net/pipermail/parenscript-devel/2009-October/000639...
(defun blah () (values 1 2 3))
(defun foo () (blah) (some-random-js-function))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
Now a call to bar returns 6, when it shouldn't. I think it might be possible to use another global variable as a flag that's set and checked by multiple-value aware PS functions, but I need to think about how to make it work.
Vladimir
On Tue, Aug 28, 2012 at 10:07 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
My guess at why your proposal won't work: It ignores knowledge of the call stack, which is practically necessary.
Imagine a form (multiple-values-bind (a b c ...) (FOO ...)). Any solution of this with global variables will take the form:
step 1: Do some stuff with global variable manipulation step 2: Call FOO step 3: Do some stuff to clean up.
Pretty much any code can execute inside FOO. This includes a few scenarios:
Scenario 1: A simple JS function, native or not, that doesn't call anything that returns multiple values. Your solution works. Scenario 2: A Parenscript function, written to manipulate your global variables properly. Your solution works. Scenario 3: A non-Parenscript function that calls a mv-returning Parenscript function.
Can your solution handle Scenario 3? My guess is it cannot because it ignores the call stack.
Just my quick guess.
- Red
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value attached to callee and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you have an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
On Tue, Aug 28, 2012 at 6:00 PM, Vladimir Sedach <vsedach@gmail.com>
Hi Daniel,
Yes, those two failures in the eval test suite are expected. CL-JavaScript doesn't have the callee.caller property for functions, which multiple value return depends on. I wasn't sure where to comment those tests out, so I left them in to remind myself to add callee.caller to CL-JavaScript (I've already talked to Marijn Haverbeke about that).
Thank you, Vladimir
On Mon, Aug 27, 2012 at 11:58 PM, Daniel Gackle <
danielgackle@gmail.com>
wrote:
I've rebased my PS LOOP extensions [1] onto the latest commit (7be9b45) and recompiled Skysheet. The generated JS looks fine. There was one glitch that I'll report separately along with a workaround. Before pushing the LOOP extensions onto master, though, I want to update any relevant PS tests. Some will fail because the LOOP output has changed quite a bit. Unfortunately I'm also seeing failures when I run the tests in 7be9b45, which is prior to any of these LOOP changes. I've pasted the output below [2]. It doesn't look like these failures are related to work in ps-loop.lisp, so I'll just ignore
wrote: them
for the time being, but Vladimir can you please comment on whether you know about them or whether there's something unexpected going on?
Daniel
[1] These are the constructs FOR..OF and MAP..TO, plus a change to FOR..ON, that I described in my email to this list on April 11. They are currently sitting in the "loop" branch. Rebasing them was nontrivial because of Boris' additions to ps-loop.lisp, but it seems to have all gone ok. Boris, if you're reading this, please look out for any regressions once I push these changes and let us know if you notice anything.
[2] Running output tests:
........................................................................................................................................................................................................................................................................................................................................................................................................................................
Did 424 checks. Pass: 424 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running package system tests: ......... Did 9 checks. Pass: 9 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running CL-JavaScript eval tests: ...........................f...............X...................... Did 66 checks. Pass: 64 (96%) Skip: 0 ( 0%) Fail: 2 ( 3%) Failure Details: -------------------------------- mv-return1 []: Unexpected Error: #<cl-js:js-condition #x30200155257D> [js] TypeError: undefined has no properties... -------------------------------- -------------------------------- dynamic-extent-function-return-values []: (funcall (if (typep #:g36204 'structure-object) #'equalp #'equal) #:g36204 (jsarray '(1 2 3))) was NIL.. --------------------------------
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you introduce an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Hi Daniel, I'm glad to be part of the discussion :) On Aug 28, 2012 8:53 PM, "Daniel Gackle" <danielgackle@gmail.com> wrote:
Hi Red,
I was hoping you'd chime in. I'll see your scenario 3 and raise you a 3a and a 3b:
Scenario 3a: A non-Parenscript function that calls a mv-returning Parenscript function but only needs its first return value;
Scenario 3b: A non-Parenscript function that calls a mv-returning Parenscript function and needs all its return values.
3a works fine as long as the MV implementation is careful to use a normal JS return to pass the first return value back to the caller. That's true both of what PS does today and of the global-var proposal.
As for 3b (the scenario, not the hacker!), seems to me this can't work at all and there's no need to support it. If you're a non-PS function then by definition you can't use PS's MULTIPLE-VALUE-BIND to access the additional return values because the MV construct only exists in PS. I suppose if you really wanted to you could manually write JS to do whatever PS does to supply those values to the caller, but then you're not really a non-PS function anymore, so much as a manually-compiled PS function.
Daniel
I wasn't clear in my original post. I'm not concerned with the non-Parenscript function's ability to receive multiple values. I'm concerned that the non-Parenscript function will interfere with the multiple value return. If a non-Parenscript function calls a Parenscript function that messes with the global variables, the non-Parenscript function could end up returning stuff unintentionally. This is because the non-Parenscript function will not manipulate the global variables to clean up the unused multiple values that are returned. Here's some code to illustrait the point: (defun ps-foo () (multiple-value-bind (a b) (bar) (+ a b))) (defun ps-fn-returns-mv () (values 1 2)) function barWorks() { return psFnReturnsMv(); } function barBreaks() { psFnReturnsMv(); // global variables for MV returning get set up and linger // return a single value: 42 return 42; } Let's assume the two Parenscript functions translate into something like this: var RETURN_VALUES = null; var MV_CALL = false; ... other global variables related to multiple values function psFoo() { // global variable stuff MV_CALL = true; RETURN_VALUES = null; var a = bar(); MV_CALL = false; var b = RETURN_VALUES ? RETURN_VALUES[0] : undefined; return a + b; } function psFnReturnsMv() { if (MV_CALL) RETURN_VALUES = [ 2 ]; return 1; } When bar = barWorks, foo will return 1 + 2, as intended. When bar = barBreaks, foo will incorrectly return 42 + 2 because the global variables were not properly cleaned up. I hope this makes sense. - Red
On Tue, Aug 28, 2012 at 8:46 PM, Red Daly <reddaly@gmail.com> wrote:
On Tue, Aug 28, 2012 at 7:20 PM, Vladimir Sedach <vsedach@gmail.com>wrote:
The counter-example to using a global variable as I remember it from the last discussion was this case:
The last discussion: http://lists.common-lisp.net/pipermail/parenscript-devel/2009-October/000639...
(defun blah () (values 1 2 3))
(defun foo () (blah) (some-random-js-function))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
Now a call to bar returns 6, when it shouldn't. I think it might be possible to use another global variable as a flag that's set and checked by multiple-value aware PS functions, but I need to think about how to make it work.
Vladimir
On Tue, Aug 28, 2012 at 10:07 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
My guess at why your proposal won't work: It ignores knowledge of the call stack, which is practically necessary.
Imagine a form (multiple-values-bind (a b c ...) (FOO ...)). Any solution of this with global variables will take the form:
step 1: Do some stuff with global variable manipulation step 2: Call FOO step 3: Do some stuff to clean up.
Pretty much any code can execute inside FOO. This includes a few scenarios:
Scenario 1: A simple JS function, native or not, that doesn't call anything that returns multiple values. Your solution works. Scenario 2: A Parenscript function, written to manipulate your global variables properly. Your solution works. Scenario 3: A non-Parenscript function that calls a mv-returning Parenscript function.
Can your solution handle Scenario 3? My guess is it cannot because it ignores the call stack.
Just my quick guess.
- Red
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value attached to callee and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you have an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
On Tue, Aug 28, 2012 at 6:00 PM, Vladimir Sedach <vsedach@gmail.com>
Hi Daniel,
Yes, those two failures in the eval test suite are expected. CL-JavaScript doesn't have the callee.caller property for functions, which multiple value return depends on. I wasn't sure where to comment those tests out, so I left them in to remind myself to add callee.caller to CL-JavaScript (I've already talked to Marijn Haverbeke about that).
Thank you, Vladimir
On Mon, Aug 27, 2012 at 11:58 PM, Daniel Gackle <
danielgackle@gmail.com>
wrote:
I've rebased my PS LOOP extensions [1] onto the latest commit (7be9b45) and recompiled Skysheet. The generated JS looks fine. There was one glitch that I'll report separately along with a workaround. Before pushing the LOOP extensions onto master, though, I want to update any relevant PS tests. Some will fail because the LOOP output has changed quite a bit. Unfortunately I'm also seeing failures when I run the tests in 7be9b45, which is prior to any of these LOOP changes. I've pasted the output below [2]. It doesn't look like
failures are related to work in ps-loop.lisp, so I'll just ignore
wrote: these them
for the time being, but Vladimir can you please comment on whether you know about them or whether there's something unexpected going on?
Daniel
[1] These are the constructs FOR..OF and MAP..TO, plus a change to FOR..ON, that I described in my email to this list on April 11. They are currently sitting in the "loop" branch. Rebasing them was nontrivial because of Boris' additions to ps-loop.lisp, but it seems to have all gone ok. Boris, if you're reading this, please look out for any regressions once I push these changes and let us know if you notice anything.
[2] Running output tests:
........................................................................................................................................................................................................................................................................................................................................................................................................................................
Did 424 checks. Pass: 424 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running package system tests: ......... Did 9 checks. Pass: 9 (100%) Skip: 0 ( 0%) Fail: 0 ( 0%) Running CL-JavaScript eval tests: ...........................f...............X...................... Did 66 checks. Pass: 64 (96%) Skip: 0 ( 0%) Fail: 2 ( 3%) Failure Details: -------------------------------- mv-return1 []: Unexpected Error: #<cl-js:js-condition #x30200155257D> [js] TypeError: undefined has no properties... -------------------------------- -------------------------------- dynamic-extent-function-return-values []: (funcall (if (typep #:g36204 'structure-object) #'equalp #'equal) #:g36204 (jsarray '(1 2 3))) was NIL.. --------------------------------
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Those test failures make sense now - thanks. Also, this comment reminded me of something:
< the callee.caller property for functions, which multiple value return depends on >
I dislike our implementation for multiple value return. Stuffing the values in callee.caller is complicated and doesn't feel right (I think I may have been the one who came up with it; it was a bad idea), plus it relies on one of the shakiest aspects if not of JS itself than certainly of the JS implementations.
A simpler way occurred to me the other day and I'd like to know where it breaks. The argument goes like this: since JS is single-threaded and functions have to return synchronously, there can be only one function return in play at any given time, therefore there can be only one multiple-return-value list at any time, therefore why not just store it in a global variable?
Say this variable is called *spillover*. Then this:
(defun blah () (values 1 2 3))
(defun callblah () (multiple-value-bind (a b c) (blah) (+ a b c)))
...might compile to:
function blah() { SPILLOVER = [2, 3]; return 1; }; function callblah() { var a = blah(); var b = SPILLOVER[1]; var c = SPILLOVER[2]; return a + b + c; };
There might be complicating factors that would make the JS more involved in practice, but I don't remember what they are.
Apart from being so much simpler, this implementation has two advantages. First, it's "morally" better in Eugenia Cheng's sense (http://cheng.staff.shef.ac.uk/morality/morality.pdf). The multiple return values don't belong to caller or callee, but to the call itself. Caller and callee are functions that persist across many calls and ought not to have information about a specific call attached to them. That's why PS is forced to add ugly code to save the previous value and restore it using a try/finally at the end.
Second, it would fix a known problem. PS breaks when you introduce an interloper like FOO here:
(defun blah () (values 1 2 3))
(defun foo () (blah))
(defun bar () (multiple-value-bind (a b c) (foo) (+ a b c)))
BAR should return 6, and does in CL, but in PS it returns 1, because FOO doesn't return multiple values, so B and C are null and "1 + null + null" is 1 in JS. But the *spillover* hack would make BAR return 6.
Could we get away with this, or what am I missing?
Daniel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Ah yes clearer. This is similar to Vladimir's case from upthread: (defun foo () (blah) (some-random-js-function)) ... except that my suggestion that the compiler figure out when BLAH isn't in a return position and do something like: function foo() { blah(); RETURN_VALUES = null; return someRandomJsFunction(); }; ... won't work in your case, i.e. when FOO is not a PS function. But I wonder whether perfect interop from JS back into PS isn't an overly ambitious a thing to promise. When language A and language B have different calling conventions and you call B from A, it's normal to have to follow some protocol to manually bridge the gap between them. In this case the protocol might be: you must either return the call to BLAH or clear RETURN_VALUES. Yeah this would be a pain and easy to forget, but it is arguably a reasonable card for PS to have to play here. That being said, it's not surprising that a simple global variable wouldn't do the trick in every case. I hope we can come up with something that does. But let's not forget that the current implementation is even more broken. It doesn't do the right thing even if BLAH is in a return position - so *none* of the cases we're talking about actually work right now. Daniel On Wed, Aug 29, 2012 at 10:16 AM, Red Daly <reddaly@gmail.com> wrote:
Hi Daniel,
I'm glad to be part of the discussion :) On Aug 28, 2012 8:53 PM, "Daniel Gackle" <danielgackle@gmail.com> wrote:
Hi Red,
I was hoping you'd chime in. I'll see your scenario 3 and raise you a 3a and a 3b:
Scenario 3a: A non-Parenscript function that calls a mv-returning Parenscript function but only needs its first return value;
Scenario 3b: A non-Parenscript function that calls a mv-returning Parenscript function and needs all its return values.
3a works fine as long as the MV implementation is careful to use a normal JS return to pass the first return value back to the caller. That's true both of what PS does today and of the global-var proposal.
As for 3b (the scenario, not the hacker!), seems to me this can't work at all and there's no need to support it. If you're a non-PS function then by definition you can't use PS's MULTIPLE-VALUE-BIND to access the additional return values because the MV construct only exists in PS. I suppose if you really wanted to you could manually write JS to do whatever PS does to supply those values to the caller, but then you're not really a non-PS function anymore, so much as a manually-compiled PS function.
Daniel
I wasn't clear in my original post. I'm not concerned with the non-Parenscript function's ability to receive multiple values. I'm concerned that the non-Parenscript function will interfere with the multiple value return.
If a non-Parenscript function calls a Parenscript function that messes with the global variables, the non-Parenscript function could end up returning stuff unintentionally. This is because the non-Parenscript function will not manipulate the global variables to clean up the unused multiple values that are returned.
Here's some code to illustrait the point:
(defun ps-foo () (multiple-value-bind (a b) (bar) (+ a b)))
(defun ps-fn-returns-mv () (values 1 2))
function barWorks() { return psFnReturnsMv(); }
function barBreaks() { psFnReturnsMv(); // global variables for MV returning get set up and linger // return a single value: 42 return 42; }
Let's assume the two Parenscript functions translate into something like this:
var RETURN_VALUES = null; var MV_CALL = false; ... other global variables related to multiple values
function psFoo() { // global variable stuff MV_CALL = true; RETURN_VALUES = null;
var a = bar(); MV_CALL = false; var b = RETURN_VALUES ? RETURN_VALUES[0] : undefined; return a + b; }
function psFnReturnsMv() { if (MV_CALL) RETURN_VALUES = [ 2 ]; return 1; }
When bar = barWorks, foo will return 1 + 2, as intended. When bar = barBreaks, foo will incorrectly return 42 + 2 because the global variables were not properly cleaned up.
I hope this makes sense.
- Red
I played around with several approaches to multiple values, and Red is correct that they all need to be stack aware. However, it turns out you don't need callee.caller for that, just first-class functions and closures. If you don't care about changing the calling convention, you don't even need that. The basic idea is that a form expecting multiple values creates a mutable array to store those values, and passes it down the stack (this is a common pattern in C code). If you don't care about function calling convention, you can just pass around a mutable array for multiple values as an implicit part of the argument list. Parenscript cares, so we need to pass that data out-of-band. We can do this by associating the function object about to be called with the multiple value array: (multiple-value-bind (x y) (foo) ...body...) prev_mv = foo.mv; // don't clobber things up the stack var values = []; foo.mv = values; x = foo(); if (values.length > 0) { y = values[0]; } foo.mv = prev_mv; ...body... Note that we just need some way to associate foo with the array. You can do that with a global table instead of setting a property on the function object. Functions that return multiple values look like: (defun foo (x y z) (values x y z)) function foo (x, y, z) { var values = arguments.callee.mv; if (values) { values[0] = y; values[1] = z; } return x; } Note that you don't need arguments.callee for a function to have a reference to itself: var foo = (function () { var self = function foo () { self.blah = whatever; ...body... }; return self; })(); That is very ugly though. multiple-value "pass-through" only happens in the case when there is an expression like "(return (some-multi-valued-function))" in the code. Since Parenscript now instruments all returns (this was not the case when the original multiple value mechanism was worked out), we can pass multiple values along like so: (defun bar () (foo) (foo)) function bar () { foo(); // first invocation, don't care about multiple values foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result; } As you can see we only give the array to functions in instances where they can potentially return multiple values. A problem arises for recursive functions (and any function objects that can appear multiple times in the stack): (defun foo (x) (if (= x 1) (values 1 2) (1+ (foo (1- x))))) foo(2) will now return multiple values, even though it shouldn't. In general, this can happen if foo calls any function x calls... a function that eventually calls foo again. This wouldn't happen if the values array was passed as an argument. One way I see to solve this problem is to add some code to any function that can potentially return multiple values: function foo (x) { var values = arguments.callee.mv; delete arguments.callee.mv; if (x === 1) { if (values) values[0] = 2; return 1; } else { return 1 + foo(x - 1); // not expecting values } } Obviously the above code will need things like unwind-protect and gensyms, etc., but does anyone see anything that's wrong with the above proposal? Vladimir On Wed, Aug 29, 2012 at 1:33 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Ah yes clearer. This is similar to Vladimir's case from upthread:
(defun foo () (blah) (some-random-js-function))
... except that my suggestion that the compiler figure out when BLAH isn't in a return position and do something like:
function foo() { blah(); RETURN_VALUES = null; return someRandomJsFunction(); };
... won't work in your case, i.e. when FOO is not a PS function.
But I wonder whether perfect interop from JS back into PS isn't an overly ambitious a thing to promise. When language A and language B have different calling conventions and you call B from A, it's normal to have to follow some protocol to manually bridge the gap between them. In this case the protocol might be: you must either return the call to BLAH or clear RETURN_VALUES. Yeah this would be a pain and easy to forget, but it is arguably a reasonable card for PS to have to play here.
That being said, it's not surprising that a simple global variable wouldn't do the trick in every case. I hope we can come up with something that does.
But let's not forget that the current implementation is even more broken. It doesn't do the right thing even if BLAH is in a return position - so *none* of the cases we're talking about actually work right now.
Daniel
On Wed, Aug 29, 2012 at 10:16 AM, Red Daly <reddaly@gmail.com> wrote:
Hi Daniel,
I'm glad to be part of the discussion :)
On Aug 28, 2012 8:53 PM, "Daniel Gackle" <danielgackle@gmail.com> wrote:
Hi Red,
I was hoping you'd chime in. I'll see your scenario 3 and raise you a 3a and a 3b:
Scenario 3a: A non-Parenscript function that calls a mv-returning Parenscript function but only needs its first return value;
Scenario 3b: A non-Parenscript function that calls a mv-returning Parenscript function and needs all its return values.
3a works fine as long as the MV implementation is careful to use a normal JS return to pass the first return value back to the caller. That's true both of what PS does today and of the global-var proposal.
As for 3b (the scenario, not the hacker!), seems to me this can't work at all and there's no need to support it. If you're a non-PS function then by definition you can't use PS's MULTIPLE-VALUE-BIND to access the additional return values because the MV construct only exists in PS. I suppose if you really wanted to you could manually write JS to do whatever PS does to supply those values to the caller, but then you're not really a non-PS function anymore, so much as a manually-compiled PS function.
Daniel
I wasn't clear in my original post. I'm not concerned with the non-Parenscript function's ability to receive multiple values. I'm concerned that the non-Parenscript function will interfere with the multiple value return.
If a non-Parenscript function calls a Parenscript function that messes with the global variables, the non-Parenscript function could end up returning stuff unintentionally. This is because the non-Parenscript function will not manipulate the global variables to clean up the unused multiple values that are returned.
Here's some code to illustrait the point:
(defun ps-foo () (multiple-value-bind (a b) (bar) (+ a b)))
(defun ps-fn-returns-mv () (values 1 2))
function barWorks() { return psFnReturnsMv(); }
function barBreaks() { psFnReturnsMv(); // global variables for MV returning get set up and linger // return a single value: 42 return 42; }
Let's assume the two Parenscript functions translate into something like this:
var RETURN_VALUES = null; var MV_CALL = false; ... other global variables related to multiple values
function psFoo() { // global variable stuff MV_CALL = true; RETURN_VALUES = null;
var a = bar(); MV_CALL = false; var b = RETURN_VALUES ? RETURN_VALUES[0] : undefined; return a + b; }
function psFnReturnsMv() { if (MV_CALL) RETURN_VALUES = [ 2 ]; return 1; }
When bar = barWorks, foo will return 1 + 2, as intended. When bar = barBreaks, foo will incorrectly return 42 + 2 because the global variables were not properly cleaned up.
I hope this makes sense.
- Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
First, your comment "they all need to be stack aware" reminds me of a point I forgot to make about the global variable idea. Obviously a naive global RETURN_VALUES array isn't going to survive cases of nested MV calls. But could the compiled PS instead keep a global RETURN_VALUES stack? i.e. a list of MV arrays that callers would push/pop as appropriate? I haven't thought about how this might work and it feels like it probably wouldn't, but I'd like to know why. < The basic idea is that a form expecting multiple values creates a mutable array to store those values, and passes it down the stack > I like the array-passing idea and agree that it probably needs to be passed out of band - but can we state explicitly why? For example, why can't we make it a hidden first argument (it couldn't very well go anywhere else because of things like &REST) and make Parenscript smart enough to add in the correct value for that hidden argument every place that function is called (i.e. pass null if the extra return values aren't to be bound, and an array to hold them if they are)? One obvious drawback is that non-PS functions wouldn't be able to call such a function normally; they'd have to know about the extra arg. What other drawbacks are there? < Note that we just need some way to associate foo with the array. You can do that with a global table instead of setting a property on the function object. > I like this idea, because everyone in the JS world is so adamant that one shouldn't mess with arguments.metablah (though arguments.callee has got to be better than arguments.callee.caller). But how would it work for lambdas? < A problem arises for recursive [including mutually recursive] functions > Right. I don't follow your example here, though, so I wonder if you can spell it out a bit further. Daniel On Thu, Aug 30, 2012 at 7:12 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
I played around with several approaches to multiple values, and Red is correct that they all need to be stack aware. However, it turns out you don't need callee.caller for that, just first-class functions and closures. If you don't care about changing the calling convention, you don't even need that.
The basic idea is that a form expecting multiple values creates a mutable array to store those values, and passes it down the stack (this is a common pattern in C code). If you don't care about function calling convention, you can just pass around a mutable array for multiple values as an implicit part of the argument list.
Parenscript cares, so we need to pass that data out-of-band. We can do this by associating the function object about to be called with the multiple value array:
(multiple-value-bind (x y) (foo) ...body...)
prev_mv = foo.mv; // don't clobber things up the stack var values = []; foo.mv = values; x = foo(); if (values.length > 0) { y = values[0]; } foo.mv = prev_mv; ...body...
Note that we just need some way to associate foo with the array. You can do that with a global table instead of setting a property on the function object.
Functions that return multiple values look like:
(defun foo (x y z) (values x y z))
function foo (x, y, z) { var values = arguments.callee.mv; if (values) { values[0] = y; values[1] = z; } return x; }
Note that you don't need arguments.callee for a function to have a reference to itself:
var foo = (function () { var self = function foo () { self.blah = whatever; ...body... }; return self; })();
That is very ugly though.
multiple-value "pass-through" only happens in the case when there is an expression like "(return (some-multi-valued-function))" in the code. Since Parenscript now instruments all returns (this was not the case when the original multiple value mechanism was worked out), we can pass multiple values along like so:
(defun bar () (foo) (foo))
function bar () { foo(); // first invocation, don't care about multiple values foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result; }
As you can see we only give the array to functions in instances where they can potentially return multiple values.
A problem arises for recursive functions (and any function objects that can appear multiple times in the stack):
(defun foo (x) (if (= x 1) (values 1 2) (1+ (foo (1- x)))))
foo(2) will now return multiple values, even though it shouldn't.
In general, this can happen if foo calls any function x calls... a function that eventually calls foo again.
This wouldn't happen if the values array was passed as an argument.
One way I see to solve this problem is to add some code to any function that can potentially return multiple values:
function foo (x) { var values = arguments.callee.mv; delete arguments.callee.mv;
if (x === 1) { if (values) values[0] = 2; return 1; } else { return 1 + foo(x - 1); // not expecting values } }
Obviously the above code will need things like unwind-protect and gensyms, etc., but does anyone see anything that's wrong with the above proposal?
Vladimir
On Wed, Aug 29, 2012 at 1:33 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Ah yes clearer. This is similar to Vladimir's case from upthread:
(defun foo () (blah) (some-random-js-function))
... except that my suggestion that the compiler figure out when BLAH isn't in a return position and do something like:
function foo() { blah(); RETURN_VALUES = null; return someRandomJsFunction(); };
... won't work in your case, i.e. when FOO is not a PS function.
But I wonder whether perfect interop from JS back into PS isn't an overly ambitious a thing to promise. When language A and language B have different calling conventions and you call B from A, it's normal to have to follow some protocol to manually bridge the gap between them. In this case the protocol might be: you must either return the call to BLAH or clear RETURN_VALUES. Yeah this would be a pain and easy to forget, but it is arguably a reasonable card for PS to have to play here.
That being said, it's not surprising that a simple global variable wouldn't do the trick in every case. I hope we can come up with something that does.
But let's not forget that the current implementation is even more broken. It doesn't do the right thing even if BLAH is in a return position - so *none* of the cases we're talking about actually work right now.
Daniel
On Wed, Aug 29, 2012 at 10:16 AM, Red Daly <reddaly@gmail.com> wrote:
Hi Daniel,
I'm glad to be part of the discussion :)
On Aug 28, 2012 8:53 PM, "Daniel Gackle" <danielgackle@gmail.com>
wrote:
Hi Red,
I was hoping you'd chime in. I'll see your scenario 3 and raise you a 3a and a 3b:
Scenario 3a: A non-Parenscript function that calls a mv-returning Parenscript function but only needs its first return
value;
Scenario 3b: A non-Parenscript function that calls a mv-returning Parenscript function and needs all its return values.
3a works fine as long as the MV implementation is careful to use a normal JS return to pass the first return value back to the caller. That's true both of what PS does today and of the global-var proposal.
As for 3b (the scenario, not the hacker!), seems to me this can't work at all and there's no need to support it. If you're a non-PS function then by definition you can't use PS's MULTIPLE-VALUE-BIND to access the additional return values because the MV construct only exists in PS. I suppose if you really wanted to you could manually write JS to do whatever PS does to supply those values to the caller, but then you're not really a non-PS function anymore, so much as a manually-compiled PS function.
Daniel
I wasn't clear in my original post. I'm not concerned with the non-Parenscript function's ability to receive multiple values. I'm concerned that the non-Parenscript function will interfere with the multiple value return.
If a non-Parenscript function calls a Parenscript function that messes with the global variables, the non-Parenscript function could end up returning stuff unintentionally. This is because the non-Parenscript function will not manipulate the global variables to clean up the unused multiple values that are returned.
Here's some code to illustrait the point:
(defun ps-foo () (multiple-value-bind (a b) (bar) (+ a b)))
(defun ps-fn-returns-mv () (values 1 2))
function barWorks() { return psFnReturnsMv(); }
function barBreaks() { psFnReturnsMv(); // global variables for MV returning get set up and linger // return a single value: 42 return 42; }
Let's assume the two Parenscript functions translate into something like this:
var RETURN_VALUES = null; var MV_CALL = false; ... other global variables related to multiple values
function psFoo() { // global variable stuff MV_CALL = true; RETURN_VALUES = null;
var a = bar(); MV_CALL = false; var b = RETURN_VALUES ? RETURN_VALUES[0] : undefined; return a + b; }
function psFnReturnsMv() { if (MV_CALL) RETURN_VALUES = [ 2 ]; return 1; }
When bar = barWorks, foo will return 1 + 2, as intended. When bar = barBreaks, foo will incorrectly return 42 + 2 because the global variables were not properly cleaned up.
I hope this makes sense.
- Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
But could the compiled PS instead keep a global RETURN_VALUES stack? i.e. a list of MV arrays that callers would push/pop as appropriate? I haven't thought about how this might work and it feels like it probably wouldn't, but I'd like to know why.
You're right that it would need to be a stack, hence the prev_mv variable in the code I posted previously. But you'd still need to associate the values with the function that returned them, and if you can do that by setting a property on the function object, why bother with global variables and tables that hold stacks? The former approach is less code.
I like the array-passing idea and agree that it probably needs to be passed out of band - but can we state explicitly why? For example, why can't we make it a hidden first argument (it couldn't very well go anywhere else because of things like &REST) and make Parenscript smart enough to add in the correct value for that hidden argument every place that function is called (i.e. pass null if the extra return values aren't to be bound, and an array to hold them if they are)?
One obvious drawback is that non-PS functions wouldn't be able to call such a function normally; they'd have to know about the extra arg. What other drawbacks are there?
You wouldn't be able to have calls to these functions before they're defined in your code.
I like this idea, because everyone in the JS world is so adamant that one shouldn't mess with arguments.metablah (though arguments.callee has got to be better than arguments.callee.caller). But how would it work for lambdas?
I tried it out in FF and CL-JS and arguments.callee always seems to be the function object, even for lambdas.
Right. I don't follow your example here, though, so I wonder if you can spell it out a bit further.
foo is a function that returns either one value or multiple values. We call foo expecting to receive multiple values. Now foo.mv is set to the multiple value array. In the course of computation, foo calls bar, which does not care about multiple values. In this second call to foo, foo.mv is still the same multiple value array. The second call to foo returns multiple values, so the multiple value array gets filled up. However, the first call to foo returns only one value. But because the multiple value array has been filled up, we get bogus multiple values. Vladimir
Daniel
On Thu, Aug 30, 2012 at 7:12 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
I played around with several approaches to multiple values, and Red is correct that they all need to be stack aware. However, it turns out you don't need callee.caller for that, just first-class functions and closures. If you don't care about changing the calling convention, you don't even need that.
The basic idea is that a form expecting multiple values creates a mutable array to store those values, and passes it down the stack (this is a common pattern in C code). If you don't care about function calling convention, you can just pass around a mutable array for multiple values as an implicit part of the argument list.
Parenscript cares, so we need to pass that data out-of-band. We can do this by associating the function object about to be called with the multiple value array:
(multiple-value-bind (x y) (foo) ...body...)
prev_mv = foo.mv; // don't clobber things up the stack var values = []; foo.mv = values; x = foo(); if (values.length > 0) { y = values[0]; } foo.mv = prev_mv; ...body...
Note that we just need some way to associate foo with the array. You can do that with a global table instead of setting a property on the function object.
Functions that return multiple values look like:
(defun foo (x y z) (values x y z))
function foo (x, y, z) { var values = arguments.callee.mv; if (values) { values[0] = y; values[1] = z; } return x; }
Note that you don't need arguments.callee for a function to have a reference to itself:
var foo = (function () { var self = function foo () { self.blah = whatever; ...body... }; return self; })();
That is very ugly though.
multiple-value "pass-through" only happens in the case when there is an expression like "(return (some-multi-valued-function))" in the code. Since Parenscript now instruments all returns (this was not the case when the original multiple value mechanism was worked out), we can pass multiple values along like so:
(defun bar () (foo) (foo))
function bar () { foo(); // first invocation, don't care about multiple values foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result; }
As you can see we only give the array to functions in instances where they can potentially return multiple values.
A problem arises for recursive functions (and any function objects that can appear multiple times in the stack):
(defun foo (x) (if (= x 1) (values 1 2) (1+ (foo (1- x)))))
foo(2) will now return multiple values, even though it shouldn't.
In general, this can happen if foo calls any function x calls... a function that eventually calls foo again.
This wouldn't happen if the values array was passed as an argument.
One way I see to solve this problem is to add some code to any function that can potentially return multiple values:
function foo (x) { var values = arguments.callee.mv; delete arguments.callee.mv;
if (x === 1) { if (values) values[0] = 2; return 1; } else { return 1 + foo(x - 1); // not expecting values } }
Obviously the above code will need things like unwind-protect and gensyms, etc., but does anyone see anything that's wrong with the above proposal?
Vladimir
On Wed, Aug 29, 2012 at 1:33 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Ah yes clearer. This is similar to Vladimir's case from upthread:
(defun foo () (blah) (some-random-js-function))
... except that my suggestion that the compiler figure out when BLAH isn't in a return position and do something like:
function foo() { blah(); RETURN_VALUES = null; return someRandomJsFunction(); };
... won't work in your case, i.e. when FOO is not a PS function.
But I wonder whether perfect interop from JS back into PS isn't an overly ambitious a thing to promise. When language A and language B have different calling conventions and you call B from A, it's normal to have to follow some protocol to manually bridge the gap between them. In this case the protocol might be: you must either return the call to BLAH or clear RETURN_VALUES. Yeah this would be a pain and easy to forget, but it is arguably a reasonable card for PS to have to play here.
That being said, it's not surprising that a simple global variable wouldn't do the trick in every case. I hope we can come up with something that does.
But let's not forget that the current implementation is even more broken. It doesn't do the right thing even if BLAH is in a return position - so *none* of the cases we're talking about actually work right now.
Daniel
On Wed, Aug 29, 2012 at 10:16 AM, Red Daly <reddaly@gmail.com> wrote:
Hi Daniel,
I'm glad to be part of the discussion :)
On Aug 28, 2012 8:53 PM, "Daniel Gackle" <danielgackle@gmail.com> wrote:
Hi Red,
I was hoping you'd chime in. I'll see your scenario 3 and raise you a 3a and a 3b:
Scenario 3a: A non-Parenscript function that calls a mv-returning Parenscript function but only needs its first return value;
Scenario 3b: A non-Parenscript function that calls a mv-returning Parenscript function and needs all its return values.
3a works fine as long as the MV implementation is careful to use a normal JS return to pass the first return value back to the caller. That's true both of what PS does today and of the global-var proposal.
As for 3b (the scenario, not the hacker!), seems to me this can't work at all and there's no need to support it. If you're a non-PS function then by definition you can't use PS's MULTIPLE-VALUE-BIND to access the additional return values because the MV construct only exists in PS. I suppose if you really wanted to you could manually write JS to do whatever PS does to supply those values to the caller, but then you're not really a non-PS function anymore, so much as a manually-compiled PS function.
Daniel
I wasn't clear in my original post. I'm not concerned with the non-Parenscript function's ability to receive multiple values. I'm concerned that the non-Parenscript function will interfere with the multiple value return.
If a non-Parenscript function calls a Parenscript function that messes with the global variables, the non-Parenscript function could end up returning stuff unintentionally. This is because the non-Parenscript function will not manipulate the global variables to clean up the unused multiple values that are returned.
Here's some code to illustrait the point:
(defun ps-foo () (multiple-value-bind (a b) (bar) (+ a b)))
(defun ps-fn-returns-mv () (values 1 2))
function barWorks() { return psFnReturnsMv(); }
function barBreaks() { psFnReturnsMv(); // global variables for MV returning get set up and linger // return a single value: 42 return 42; }
Let's assume the two Parenscript functions translate into something like this:
var RETURN_VALUES = null; var MV_CALL = false; ... other global variables related to multiple values
function psFoo() { // global variable stuff MV_CALL = true; RETURN_VALUES = null;
var a = bar(); MV_CALL = false; var b = RETURN_VALUES ? RETURN_VALUES[0] : undefined; return a + b; }
function psFnReturnsMv() { if (MV_CALL) RETURN_VALUES = [ 2 ]; return 1; }
When bar = barWorks, foo will return 1 + 2, as intended. When bar = barBreaks, foo will incorrectly return 42 + 2 because the global variables were not properly cleaned up.
I hope this makes sense.
- Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
(I get bounced if the reply includes too much of prev thread, so am truncating to immediate predecessor.) I think we should avoid foo.mv and just pass the array as an implicit arg (see below). First a couple responses: < I tried it out in FF and CL-JS and arguments.callee always seems to be the function object, even for lambdas. > We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme. < In this second call to foo, foo.mv is still the same multiple value array. > Ok, I get it now. Interestingly, this is the stack problem all over again. Foo.mv may not be global but it's global per-function, so it fails to work when foo is in the stack more than once. I still strongly favour your array-passing proposal - namely that when compiling a MULTIPLE-VALUE-BIND, PS would pass an extra argument in the function call: a mutable array to hold any extra return values; and when compiling a VALUES, PS would generate code to check whether that array was passed and populate it if so. But I said something that I now believe to be false: that such an implicit argument "couldn't very well go anywhere [other than the first position] because of things like &REST". Maybe we can sneak it into the last position without disturbing anybody else. Let PS declare a global sentinel value: var MV_SENTINEL = {} This can't accidentally be equal to anything else so we can use it to unambiguously indicate MV-ness; specifically, we can pass it to indicate that the succeeding arg is an MV array. A form like: (multiple-value-bind (x y) (foo a b) ...body...) could compile to something like: var values = []; x = foo(a, b, MV_SENTINEL, values); if (values.length > 0) { y = values[0]; } And where the values are returned: (defun foo (x y) (values x y)) could compile to something like: function foo (x, y) { var values = (arguments[2] === MV_SENTINEL) ? arguments[3] : undefined; if (values) { values[0] = y; } return x; } This is clean and stack-friendly. Callers that don't care about MV don't have to deal with it (non-PS functions in particular are fine); callers that want MV take on implicit arguments to get it. It's stateless and seems easy to implement. No try/finally and no hacking callee or caller. At first I thought this wouldn't work because it would collide with &REST and &KEY, which already interpret the arguments list. But why should it? It's PS that generates the code to bind arguments to &REST and &KEY params. For functions that include both a VALUES and a &REST/&KEY, PS can just generate slightly smarter code that checks for MV_SENTINEL and, if it's present, avoids binding it or its successor as part of &REST or &KEY. Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be pretty powerful. Where does this break? Daniel p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have: (defun foo (x y) (values x y (blah)) but a caller doesn't want that third return value: (multiple-value-bind (x y) (foo a b) ...body...) The caller could signal how much it wants like this: var values = [true]; // fill up to number of values desired, in this case 1 x = foo(a, b, MV_SENTINEL, values); if (typeof values[0] !== undefined) { y = values[0]; } ... and foo could do something like: function foo (x, y) { var values = (arguments[2] === MV_SENTINEL) ? arguments[3] : undefined; if (values) { values[0] = values[0] ? y : undefined; values[1] = values[1] ? blah() : undefined; } return x; } On Sat, Sep 1, 2012 at 3:48 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
But could the compiled PS instead keep a global RETURN_VALUES stack? i.e. a list of MV arrays that callers would push/pop as appropriate? I haven't thought about how this might work and it feels like it probably wouldn't, but I'd like to know why.
You're right that it would need to be a stack, hence the prev_mv variable in the code I posted previously. But you'd still need to associate the values with the function that returned them, and if you can do that by setting a property on the function object, why bother with global variables and tables that hold stacks? The former approach is less code.
I like the array-passing idea and agree that it probably needs to be passed out of band - but can we state explicitly why? For example, why can't we make it a hidden first argument (it couldn't very well go anywhere else because of things like &REST) and make Parenscript smart enough to add in the correct value for that hidden argument every place that function is called (i.e. pass null if the extra return values aren't to be bound, and an array to hold them if they are)?
One obvious drawback is that non-PS functions wouldn't be able to call such a function normally; they'd have to know about the extra arg. What other drawbacks are there?
You wouldn't be able to have calls to these functions before they're defined in your code.
I like this idea, because everyone in the JS world is so adamant that one shouldn't mess with arguments.metablah (though arguments.callee has got to be better than arguments.callee.caller). But how would it work for lambdas?
I tried it out in FF and CL-JS and arguments.callee always seems to be the function object, even for lambdas.
Right. I don't follow your example here, though, so I wonder if you can spell it out a bit further.
foo is a function that returns either one value or multiple values. We call foo expecting to receive multiple values. Now foo.mv is set to the multiple value array. In the course of computation, foo calls bar, which does not care about multiple values. In this second call to foo, foo.mv is still the same multiple value array. The second call to foo returns multiple values, so the multiple value array gets filled up. However, the first call to foo returns only one value. But because the multiple value array has been filled up, we get bogus multiple values.
Vladimir
We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme.
For my first prototype for the new MV mechanism, that's what I thought and used gensyms. But then I tried foo[<function object>] and that works in both FF and CL-JS. But looking at ECMAScript, property identifiers do indeed have to be JavaScript String objects (http://ecma-international.org/ecma-262/5.1/#sec-8.10).
Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be pretty powerful.
Where does this break?
It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever.
p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have:
In general you have to evaluate all the expressions given to values for their side-effects, so this would only save a few assignments. Vladimir
< It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever. > True, but PS already imposes Lisp lambda-list semantics on the arguments pseudo-array. Functions that manipulate it (other than for just passing arguments on) will break on things like &key params if they don't interpret them correctly. So I'm not sure that the price here isn't one we've already had to pay, and if that's true then it hasn't turned out to be much of a problem. I'd be interested to see examples that break on the implicit MV arg that don't already break on &key. Passing the arguments pseudo-array on, for example, shouldn't break. It just throws the responsibility for parsing the MV arg on to somebody else, which in non-MV-aware cases will typically ignore it and in MV-aware cases will handle it according to the protocol. It strikes me that the implicit MV arg is a generalization of the &key mechanism where the key is not a symbol (string) but a sentinel object. The big break with current practice is that it's hidden rather than specified by the user at the source level. Makes me wonder if a different syntax than MULTIPLE-VALUE-BIND might help - something to make the underlying mechanism less unexpected... To be sure, it's a hack and not at all what one would do with proper access to the internals. But I wonder if it's as good as we're likely to get by way of a correct implementation. On Sun, Sep 2, 2012 at 6:28 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme.
For my first prototype for the new MV mechanism, that's what I thought and used gensyms. But then I tried foo[<function object>] and that works in both FF and CL-JS. But looking at ECMAScript, property identifiers do indeed have to be JavaScript String objects (http://ecma-international.org/ecma-262/5.1/#sec-8.10).
Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be pretty powerful.
Where does this break?
It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever.
p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have:
In general you have to evaluate all the expressions given to values for their side-effects, so this would only save a few assignments.
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
I'd be interested to see examples that break on the implicit MV arg that don't already break on &key. Passing the arguments pseudo-array on, for example, shouldn't break. It just throws the responsibility for parsing the MV arg on to somebody else, which in non-MV-aware cases will typically ignore it and in MV-aware cases will handle it according to the protocol.
If you call a function FOO with &key parameters you're expecting it to be a PS function, and if function FOO calls other functions they don't care. You can make the same argument for calling a function BAR expecting multiple values. Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns, which might or might not lead to subtle bugs. So I am really opposed to this approach. Vladimir
It strikes me that the implicit MV arg is a generalization of the &key mechanism where the key is not a symbol (string) but a sentinel object. The big break with current practice is that it's hidden rather than specified by the user at the source level. Makes me wonder if a different syntax than MULTIPLE-VALUE-BIND might help - something to make the underlying mechanism less unexpected...
To be sure, it's a hack and not at all what one would do with proper access to the internals. But I wonder if it's as good as we're likely to get by way of a correct implementation.
On Sun, Sep 2, 2012 at 6:28 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme.
For my first prototype for the new MV mechanism, that's what I thought and used gensyms. But then I tried foo[<function object>] and that works in both FF and CL-JS. But looking at ECMAScript, property identifiers do indeed have to be JavaScript String objects (http://ecma-international.org/ecma-262/5.1/#sec-8.10).
Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be pretty powerful.
Where does this break?
It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever.
p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have:
In general you have to evaluate all the expressions given to values for their side-effects, so this would only save a few assignments.
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
You've convinced me about subtle bugs. But I would like to get clear about the root difficulty. I don't think this quite captures it: < Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns > ... because if BAR *returns* that other function's value(s) then we actually want whatever it puts in the MV array - yes? That is, in the following: (defun foo () (multiple-value-bind (a b) (bar) // ignore a b)) (defun bar () (apply #'baz arguments)) (defun baz () (values 10 20)) ... FOO should return 20, so it's good that the MV semantics pass through BAR to BAZ. The problem comes when BAR *doesn't* return its call to BAZ, and BAZ has populated the MV array with bogus values: (defun bar () (apply #'baz arguments) nil) Now FOO should clearly not return 20. Does this illustrate what you are objecting to? What I find interesting is how the different implementations we've been discussing all seem to break on the same thing. Correct compilation of MVR requires distinguishing between function calls in a tail position (where MV passthrough is wanted) and those not in a tail position (where MV passthrough is a bug). That is, the granularity within which MV state should be shared and outside which it must not leak is: a chain of function calls in the tail position. That explains why the proposed implementations so far are incorrect: implicit argument as described above, MV_RETURNS (because MV state is global), and foo.mv (because MV state is per-function [1]). None of them works at this granularity. So two questions: 1) is this explanation of the difficulty correct? 2) Is there a way for PS to ensure that MV state can't leak outside a chain of tail calls? A naive way to do it would be to generate code to clear MV state after every non-tail call. Daniel [1] With respect to the foo.mv implementation, we had talked about things breaking on recursion. It's worth noting that recursion and MV work fine together as long as the MV calls are tail calls. e.g. BAR returns 6 here: (defun foo (list) (cond ((null list) (values 1 2 3)) (t (foo (cdr list))))) (defun bar () (multiple-value-bind (a b c) (foo '(blah blah)) (+ a b c))) On Mon, Sep 3, 2012 at 8:55 AM, Vladimir Sedach <vsedach@gmail.com> wrote:
I'd be interested to see examples that break on the implicit MV arg that don't already break on &key. Passing the arguments pseudo-array on, for example, shouldn't break. It just throws the responsibility for parsing the MV arg on to somebody else, which in non-MV-aware cases will typically ignore it and in MV-aware cases will handle it according to the protocol.
If you call a function FOO with &key parameters you're expecting it to be a PS function, and if function FOO calls other functions they don't care. You can make the same argument for calling a function BAR expecting multiple values. Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns, which might or might not lead to subtle bugs. So I am really opposed to this approach.
Vladimir
It strikes me that the implicit MV arg is a generalization of the &key mechanism where the key is not a symbol (string) but a sentinel object. The big break with current practice is that it's hidden rather than specified by the user at the source level. Makes me wonder if a different syntax than MULTIPLE-VALUE-BIND might help - something to make the underlying mechanism less unexpected...
To be sure, it's a hack and not at all what one would do with proper access to the internals. But I wonder if it's as good as we're likely to get by way of a correct implementation.
On Sun, Sep 2, 2012 at 6:28 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme.
For my first prototype for the new MV mechanism, that's what I thought and used gensyms. But then I tried foo[<function object>] and that works in both FF and CL-JS. But looking at ECMAScript, property identifiers do indeed have to be JavaScript String objects (http://ecma-international.org/ecma-262/5.1/#sec-8.10).
Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be pretty powerful.
Where does this break?
It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever.
p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have:
In general you have to evaluate all the expressions given to values for their side-effects, so this would only save a few assignments.
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Um, and duh: implicit arg technique fails to do any passthrough at all in most cases: (defun foo () (multiple-value-bind (a b) (bar) // ignore a b)) (defun bar () (baz)) (defun baz () (values 10 20)) MV semantics ought to hop along the chain FOO->BAR->BAZ and back, but here BAR does nothing to forward the arguments list, so BAZ never gets an implicit MV array to populate. In other words, implicit arg in general is good for one call only, which is not good enough. I'm surprised I didn't see this before. It only became obvious to me after that last point about tail calls. Daniel On Mon, Sep 3, 2012 at 12:58 PM, Daniel Gackle <danielgackle@gmail.com>wrote:
You've convinced me about subtle bugs. But I would like to get clear about the root difficulty. I don't think this quite captures it:
< Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns >
... because if BAR *returns* that other function's value(s) then we actually want whatever it puts in the MV array - yes? That is, in the following:
(defun foo () (multiple-value-bind (a b) (bar) // ignore a b))
(defun bar () (apply #'baz arguments))
(defun baz () (values 10 20))
... FOO should return 20, so it's good that the MV semantics pass through BAR to BAZ. The problem comes when BAR *doesn't* return its call to BAZ, and BAZ has populated the MV array with bogus values:
(defun bar () (apply #'baz arguments) nil)
Now FOO should clearly not return 20. Does this illustrate what you are objecting to?
What I find interesting is how the different implementations we've been discussing all seem to break on the same thing. Correct compilation of MVR requires distinguishing between function calls in a tail position (where MV passthrough is wanted) and those not in a tail position (where MV passthrough is a bug). That is, the granularity within which MV state should be shared and outside which it must not leak is: a chain of function calls in the tail position.
That explains why the proposed implementations so far are incorrect: implicit argument as described above, MV_RETURNS (because MV state is global), and foo.mv (because MV state is per-function [1]). None of them works at this granularity.
So two questions: 1) is this explanation of the difficulty correct? 2) Is there a way for PS to ensure that MV state can't leak outside a chain of tail calls? A naive way to do it would be to generate code to clear MV state after every non-tail call.
Daniel
[1] With respect to the foo.mv implementation, we had talked about things breaking on recursion. It's worth noting that recursion and MV work fine together as long as the MV calls are tail calls. e.g. BAR returns 6 here:
(defun foo (list) (cond ((null list) (values 1 2 3)) (t (foo (cdr list)))))
(defun bar () (multiple-value-bind (a b c) (foo '(blah blah)) (+ a b c)))
On Mon, Sep 3, 2012 at 8:55 AM, Vladimir Sedach <vsedach@gmail.com> wrote:
I'd be interested to see examples that break on the implicit MV arg that don't already break on &key. Passing the arguments pseudo-array on, for example, shouldn't break. It just throws the responsibility for parsing the MV arg on to somebody else, which in non-MV-aware cases will typically ignore it and in MV-aware cases will handle it according to the protocol.
If you call a function FOO with &key parameters you're expecting it to be a PS function, and if function FOO calls other functions they don't care. You can make the same argument for calling a function BAR expecting multiple values. Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns, which might or might not lead to subtle bugs. So I am really opposed to this approach.
Vladimir
It strikes me that the implicit MV arg is a generalization of the &key mechanism where the key is not a symbol (string) but a sentinel object. The big break with current practice is that it's hidden rather than specified by the user at the source level. Makes me wonder if a different syntax than MULTIPLE-VALUE-BIND might help - something to make the underlying mechanism less unexpected...
To be sure, it's a hack and not at all what one would do with proper access to the internals. But I wonder if it's as good as we're likely to get by way of a correct implementation.
On Sun, Sep 2, 2012 at 6:28 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme.
For my first prototype for the new MV mechanism, that's what I thought and used gensyms. But then I tried foo[<function object>] and that works in both FF and CL-JS. But looking at ECMAScript, property identifiers do indeed have to be JavaScript String objects (http://ecma-international.org/ecma-262/5.1/#sec-8.10).
Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be
pretty
powerful.
Where does this break?
It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever.
p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have:
In general you have to evaluate all the expressions given to values for their side-effects, so this would only save a few assignments.
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
That's the issue I was refering to when talking about instrumenting returns in my earlier post. Some variation of this code is going to be needed, whether it passes on the multiple values array as an extra argument or as a property on the function object: (defun bar () (foo) (foo)) function bar () { foo(); // first invocation, don't care about multiple values foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result; } Vladimir On Mon, Sep 3, 2012 at 4:36 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
Um, and duh: implicit arg technique fails to do any passthrough at all in most cases:
(defun foo () (multiple-value-bind (a b) (bar) // ignore a b))
(defun bar () (baz))
(defun baz () (values 10 20))
MV semantics ought to hop along the chain FOO->BAR->BAZ and back, but here BAR does nothing to forward the arguments list, so BAZ never gets an implicit MV array to populate. In other words, implicit arg in general is good for one call only, which is not good enough.
I'm surprised I didn't see this before. It only became obvious to me after that last point about tail calls.
Daniel
On Mon, Sep 3, 2012 at 12:58 PM, Daniel Gackle <danielgackle@gmail.com> wrote:
You've convinced me about subtle bugs. But I would like to get clear about the root difficulty. I don't think this quite captures it:
< Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns >
... because if BAR *returns* that other function's value(s) then we actually want whatever it puts in the MV array - yes? That is, in the following:
(defun foo () (multiple-value-bind (a b) (bar) // ignore a b))
(defun bar () (apply #'baz arguments))
(defun baz () (values 10 20))
... FOO should return 20, so it's good that the MV semantics pass through BAR to BAZ. The problem comes when BAR *doesn't* return its call to BAZ, and BAZ has populated the MV array with bogus values:
(defun bar () (apply #'baz arguments) nil)
Now FOO should clearly not return 20. Does this illustrate what you are objecting to?
What I find interesting is how the different implementations we've been discussing all seem to break on the same thing. Correct compilation of MVR requires distinguishing between function calls in a tail position (where MV passthrough is wanted) and those not in a tail position (where MV passthrough is a bug). That is, the granularity within which MV state should be shared and outside which it must not leak is: a chain of function calls in the tail position.
That explains why the proposed implementations so far are incorrect: implicit argument as described above, MV_RETURNS (because MV state is global), and foo.mv (because MV state is per-function [1]). None of them works at this granularity.
So two questions: 1) is this explanation of the difficulty correct? 2) Is there a way for PS to ensure that MV state can't leak outside a chain of tail calls? A naive way to do it would be to generate code to clear MV state after every non-tail call.
Daniel
[1] With respect to the foo.mv implementation, we had talked about things breaking on recursion. It's worth noting that recursion and MV work fine together as long as the MV calls are tail calls. e.g. BAR returns 6 here:
(defun foo (list) (cond ((null list) (values 1 2 3)) (t (foo (cdr list)))))
(defun bar () (multiple-value-bind (a b c) (foo '(blah blah)) (+ a b c)))
On Mon, Sep 3, 2012 at 8:55 AM, Vladimir Sedach <vsedach@gmail.com> wrote:
I'd be interested to see examples that break on the implicit MV arg that don't already break on &key. Passing the arguments pseudo-array on, for example, shouldn't break. It just throws the responsibility for parsing the MV arg on to somebody else, which in non-MV-aware cases will typically ignore it and in MV-aware cases will handle it according to the protocol.
If you call a function FOO with &key parameters you're expecting it to be a PS function, and if function FOO calls other functions they don't care. You can make the same argument for calling a function BAR expecting multiple values. Except as you point out the multiple-values array will now be passed on to any functions whose value BAR returns, which might or might not lead to subtle bugs. So I am really opposed to this approach.
Vladimir
It strikes me that the implicit MV arg is a generalization of the &key mechanism where the key is not a symbol (string) but a sentinel object. The big break with current practice is that it's hidden rather than specified by the user at the source level. Makes me wonder if a different syntax than MULTIPLE-VALUE-BIND might help - something to make the underlying mechanism less unexpected...
To be sure, it's a hack and not at all what one would do with proper access to the internals. But I wonder if it's as good as we're likely to get by way of a correct implementation.
On Sun, Sep 2, 2012 at 6:28 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
We may have crossed a wire here. When you said, "You can do that with a global table instead of setting a property on the function object," I thought you had in mind a global table keyed by function *name*, which is why I asked about lambdas since they have no names. JS won't let you use a function object as a key so one would have to concoct some naming scheme.
For my first prototype for the new MV mechanism, that's what I thought and used gensyms. But then I tried foo[<function object>] and that works in both FF and CL-JS. But looking at ECMAScript, property identifiers do indeed have to be JavaScript String objects (http://ecma-international.org/ecma-262/5.1/#sec-8.10).
Moreover, if it worked for MV, the sentinel idea could be used to tag anything else we wanted into the call. Seems like that could be pretty powerful.
Where does this break?
It wouldn't work for calling JavaScript functions that used the arguments pseudo-array, either for arbitrary arity or for passing arguments on. There's lots of JS functions around that do things like foo.apply(null, slice(arguments, x)) or whatever.
p.s. There's also a way to communicate exactly how many return values are desired, short-circuiting any computation that might be needed to generate the full VALUES list. (I think this point is independent of the above idea but I'll adapt the same examples.) Suppose we have:
In general you have to evaluate all the expressions given to values for their side-effects, so this would only save a few assignments.
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
I have a question about Vladimir's proposal upthread (from Aug 30) for passing implicit mv arrays. The idea is to pass MV by setting FOO.MV before calling FOO, and to read MV passed to oneself as ARGUMENTS.CALLEE.MV. As we've been discussing, correct MVR must: (1) enable passthrough where it's wanted, and (2) suppress passthrough where it's not wanted. This proposal gets (1) by instrumenting code to pass MV everywhere it might be needed, and (2) by simply not instrumenting where it isn't wanted [*]. Here's Vladimir's original example (with comment edited): (defun foo (x y z) (values x y z)) (defun bar () (foo) (foo)) function bar () { foo(); // no instrumentation since passthrough is not wanted foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result; } But passthrough is transitive, so we can't just instrument tail calls to FOO [**]. We also have to instrument tail calls to things that tail call FOO, like BAZ here: (defun foo (x y z) (values x y z)) (defun baz () (foo)) (defun bar () (foo) (baz)) Given this transitivity, how do you distinguish which tail calls are potentially-multi-valued in order to instrument them? Keep in mind that passthrough also has to work in cases like: (defun baz2 (fn) (apply fn '(1 2 3)) (defun baz3 () (funcall *fn* 1 2 3)) ... whenever the calls to FN and *FN* are potentially-multi-valued. The only easy implementation I can think of is to mv-instrument every tail call in the program, but that is surely overkill. Daniel [*] There's also the recursive case we've been discussing, where you might both want the instrumentation and not want it -- but since my question doesn't depend on that complication, let's just ignore it here. [**] Just to be clear, all I mean by "tail call" is any expression "(return (some-function))" once PS is done adding in all the implicit returns.
Hi Dan,
The only easy implementation I can think of is to mv-instrument every tail call in the program, but that is surely overkill.
Yes, that's currently the only way I see to do it. This issue is orthogonal to how the multiple values are passed - in principle I could have implemented the multiple value passing instrumentation right after adding the return instrumentation code, but it didn't occur to me at the time. So to clarify, my proposal has two separate components: 1. Pass multiple values on the callee function object 2. Instrument every tail call Vladimir
OK, we're on the same wavelength. I'm glad you said "This issue [i.e. passthrough] is orthogonal to how the multiple values are passed" -- that's the same conclusion I've come to. Now I wonder if we can fix the global variable (GV) design. Naive GV had no way to suppress incorrect passthrough. But if it can be corrected, a GV implementation would probably be simpler and more efficient than foo.mv/arguments.callee.mv. The foo.mv way suppresses passthrough by default and turns it on by instrumenting tail calls. What about the inverse? Use a GV to enable passthrough by default and turn it off by instrumenting *non* tail calls. Something like: (defun foo (x y z) (values x y z)) (defun baz () (foo)) (defun bar () (foo) (baz)) var EXTRAS = null; // global variable for MV returns function foo(x, y, z) { EXTRAS = [y, z]; return x; } function baz() { return foo(); // passthrough enabled by default } function bar () { foo(); EXTRAS = null; // instrumentation to suppress passthrough return baz(); // transitive passthrough is free } You'd want "EXTRAS = null" after each MULTIPLE-VALUE-BIND as well, but maybe you get this for free since the function call in a MULTIPLE-VALUE-BIND is by definition not a tail call. We can summarize this proposal similarly to the other one: 1. Pass multiple values using a global variable 2. Instrument every non-tail call. It may still be too naive but it at least fixes the examples that broke GV earlier in the thread. So, is there an example that works in foo.mv but fails in GV as described above? Dan p.s. I also wonder if the recursion difficulty disappears now that we understand passthrough better. The example was: (defun foo (x) (if (= x 1) (values 1 2) (1+ (foo (1- x))))) In both the GV and foo.mv designs, shouldn't foo(2) do the right thing now? That recursion is a non-tail call, so GV would suppress MV passthrough and foo.mv wouldn't enable it. On Fri, Sep 7, 2012 at 12:46 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
Hi Dan,
The only easy implementation I can think of is to mv-instrument every tail call in the program, but that is surely overkill.
Yes, that's currently the only way I see to do it. This issue is orthogonal to how the multiple values are passed - in principle I could have implemented the multiple value passing instrumentation right after adding the return instrumentation code, but it didn't occur to me at the time.
So to clarify, my proposal has two separate components: 1. Pass multiple values on the callee function object 2. Instrument every tail call
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
You'd want "EXTRAS = null" after each MULTIPLE-VALUE-BIND as well, but maybe you get this for free since the function call in a MULTIPLE-VALUE-BIND is by definition not a tail call.
We can summarize this proposal similarly to the other one:
1. Pass multiple values using a global variable 2. Instrument every non-tail call.
It may still be too naive but it at least fixes the examples that broke GV earlier in the thread. So, is there an example that works in foo.mv but fails in GV as described above?
I see a problem in that you'd have to instrument not just return statments, but any kind of control flow that goes up the stack - so you'd have to put the suppression code at the end of every function definition, and also wrap throws. So this wouldn't work if you call a JS function expecting multiple values, and that JS function happens to call a PS function that wants to return multiple values.
p.s. I also wonder if the recursion difficulty disappears now that we understand passthrough better. The example was:
(defun foo (x) (if (= x 1) (values 1 2) (1+ (foo (1- x)))))
In both the GV and foo.mv designs, shouldn't foo(2) do the right thing now? That recursion is a non-tail call, so GV would suppress MV passthrough and foo.mv wouldn't enable it.
You're right. Good point. Vladimir
I don't understand this bit. Can you give an example?
So this wouldn't work if you call a JS function expecting multiple values, and that JS function happens to call a PS function that wants to return multiple values.
In any case, interoperability with non-PS programs is a separate issue- as far as I can tell, neither the GV nor foo.mv designs can achieve it.
I see a problem in that you'd have to instrument not just return statments, but any kind of control flow that goes up the stack - so you'd have to put the suppression code at the end of every function definition, and also wrap throws.
I haven't come up with any examples of this yet and would like to see one. It seems the question is, how might the GV design fail to clean up dangling MV state? There are three ways for MV state to flow in GV: (1) non-tail calls that clear MV; (2) non-tail calls that bind MV and then clear it; (3) tail calls that let MV pass through. The only instrumentation that GV adds is to clear MV in cases (1) and (2). It doesn't instrument return statements. I think one can show that (1) and (2) can't fail in a way that leaves any dangling MV. Consider (2). The JS looks something like: var temp = firstResult(); MV = [secondResult()]; return temp; // in caller var blah = undefined; if (MV) { blah = MV[0]; } MV = null; There's no way for any of this code to fail except in firstResult() or secondResult(), in which case MV never gets assigned and no state can dangle. That answers case (2). For case (1) just remove the code that binds MV. That leaves (3), tail call passthrough. Can this case fail? I was going to say it could, but now I'm not sure. Consider this example: (defun foo (x y) (blah1) (values x y)) (defun baz () (blah2) (foo)) (defun bar () (blah3) (multiple-value-bind (a b) (baz) (blah4 a b))) If an error is thrown from blah1, blah2, or blah3, no MV is assigned. If an error is thrown from blah4, MV has already been cleared. The argument is that the tail-call-ness of (FOO) inside BAZ doesn't leave any space for failures to "squeeze in". One example isn't the general case, obviously, but having gotten this far I'd be intrigued to see an example of where GV can leave dangling state (apart from interop with non-PS code, which as I said I think both designs fail at, and which I suspect is impossible to do perfectly). It's worth noting that foo.mv might have a problem with errors too: foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result; If foo() throws an error, there is a dangling foo.mv. Might that be harmless? If it isn't, you need something like: foo.mv = arguments.callee.mv; try { return foo(); } finally { delete foo.mv; } ... around every tail call in the program. The possible performance hit is worrisome. That's on top of the possible performance hit of arguments.callee.mv, which is the kind of thing that causes V8 to turn off optimizations. Daniel On Fri, Sep 7, 2012 at 7:00 PM, Vladimir Sedach <vsedach@gmail.com> wrote:
You'd want "EXTRAS = null" after each MULTIPLE-VALUE-BIND as well, but maybe you get this for free since the function call in a MULTIPLE-VALUE-BIND is by definition not a tail call.
We can summarize this proposal similarly to the other one:
1. Pass multiple values using a global variable 2. Instrument every non-tail call.
It may still be too naive but it at least fixes the examples that broke GV earlier in the thread. So, is there an example that works in foo.mv but fails in GV as described above?
I see a problem in that you'd have to instrument not just return statments, but any kind of control flow that goes up the stack - so you'd have to put the suppression code at the end of every function definition, and also wrap throws. So this wouldn't work if you call a JS function expecting multiple values, and that JS function happens to call a PS function that wants to return multiple values.
p.s. I also wonder if the recursion difficulty disappears now that we understand passthrough better. The example was:
(defun foo (x) (if (= x 1) (values 1 2) (1+ (foo (1- x)))))
In both the GV and foo.mv designs, shouldn't foo(2) do the right thing now? That recursion is a non-tail call, so GV would suppress MV passthrough and foo.mv wouldn't enable it.
You're right. Good point.
Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
In any case, interoperability with non-PS programs is a separate issue- as far as I can tell, neither the GV nor foo.mv designs can achieve it.
With the function object approach, JS functions will always return exactly one value. IMO, having them pass through multiple values would be the wrong thing.
I see a problem in that you'd have to instrument not just return statments, but any kind of control flow that goes up the stack - so you'd have to put the suppression code at the end of every function definition, and also wrap throws.
I haven't come up with any examples of this yet and would like to see one. It seems the question is, how might the GV design fail to clean up dangling MV state? There are three ways for MV state to flow in GV:
(1) non-tail calls that clear MV; (2) non-tail calls that bind MV and then clear it; (3) tail calls that let MV pass through.
Other exit points from a function are throws and control reaching the end of the function. So both are going to have to get instrumented.
It's worth noting that foo.mv might have a problem with errors too:
foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result;
If foo() throws an error, there is a dangling foo.mv. Might that be harmless? If it isn't, you need something like:
foo.mv = arguments.callee.mv; try { return foo(); } finally { delete foo.mv; }
... around every tail call in the program. The possible performance hit is worrisome. That's on top of the possible performance hit of arguments.callee.mv, which is the kind of thing that causes V8 to turn off optimizations.
Yeah, I just finished up coding the function object approach, and the generated code gets ugly. I don't see a way around instrumenting tail calls if you want to implement multiple value pass-through correctly. I do see a way to combine the global variable and function object approach to simplify things: Since there's only one way the values can travel up the stack, we can use a single global variable to store the multiple values with the function object that returned them. That way multiple-value-bind can check the array, and if it has values from any but the expected function object, we can just ignore them. To pass through multiple values, every tail call now has to check the global variable to see if multiple values got returned by the tail function. If so, it just replaces the function object they're tagged with with itself. I can see that approach failing for a recursive function that sets multiple values in one of its sub-invocations, but doesn't return multiple values to whoever originally called it expecting multiple values. Vladimir
Ok, just to give an example, under the proposed global+funobj scheme, a tail call to foo() will look like: __MV_FLAGS = __MV_ARRAY = []; var funobj_gensym = foo; var result_gensym = funobj_gensym(); __MV_FLAGS = __MV_FLAGS === funobj_gensym? arguments.callee : null; return result_gensym; funobj_gensym can be optimized out if we know foo is a symbol, and not a lambda or a function-valued expression. This still leaves the following case broken: (progn (defun foo (x) (try (if (= 0 x) (values 1 2 3) (bar)) (:catch (e) 27))) (defun bar () (foo 0) (throw 13)) (multiple-value-bind (a b c) (foo 1) (list a b c))) foo calls bar calls foo. The last call to foo return multiple values, which in the global variable solution get put in a global. bar never clears the global variable. the first call to foo catches the throw from bar, and returns 27. The problem is the global variable holding multiple values is tagged with foo's function object, so m-v-b thinks foo returned (values 27 2 3). Right now that test case works with the original funobj-only scheme. I've pushed the code to the PS repository so people can play with it and it's in the history, but I won't be putting that code into the upcoming release. To fix the problem above in the global var scheme, bar is going to have to clear multiple values not just on tail calls, but on all other kinds of returns. If you want to play around with a complete multiple-value-bind implementation that does passthrough correctly, go to ea9044693d14c8 in HEAD. What I'm planning to do next is to roll back the complete solution to not do pass-through - it has improvements like multiple values from nested block returns, and not using arguments.callee.caller, that the original multiple value solution lacks. If anyone has ideas about making passthrough work without lots of code overhead for tail calls and function prologues, I'd like to hear it. Otherwise I'm going to leave pass-through out of the upcoming release. Vladimir On Wed, Sep 12, 2012 at 3:12 AM, Vladimir Sedach <vsedach@gmail.com> wrote:
In any case, interoperability with non-PS programs is a separate issue- as far as I can tell, neither the GV nor foo.mv designs can achieve it.
With the function object approach, JS functions will always return exactly one value. IMO, having them pass through multiple values would be the wrong thing.
I see a problem in that you'd have to instrument not just return statments, but any kind of control flow that goes up the stack - so you'd have to put the suppression code at the end of every function definition, and also wrap throws.
I haven't come up with any examples of this yet and would like to see one. It seems the question is, how might the GV design fail to clean up dangling MV state? There are three ways for MV state to flow in GV:
(1) non-tail calls that clear MV; (2) non-tail calls that bind MV and then clear it; (3) tail calls that let MV pass through.
Other exit points from a function are throws and control reaching the end of the function. So both are going to have to get instrumented.
It's worth noting that foo.mv might have a problem with errors too:
foo.mv = arguments.callee.mv; var result = foo(); delete foo.mv; return result;
If foo() throws an error, there is a dangling foo.mv. Might that be harmless? If it isn't, you need something like:
foo.mv = arguments.callee.mv; try { return foo(); } finally { delete foo.mv; }
... around every tail call in the program. The possible performance hit is worrisome. That's on top of the possible performance hit of arguments.callee.mv, which is the kind of thing that causes V8 to turn off optimizations.
Yeah, I just finished up coding the function object approach, and the generated code gets ugly.
I don't see a way around instrumenting tail calls if you want to implement multiple value pass-through correctly.
I do see a way to combine the global variable and function object approach to simplify things:
Since there's only one way the values can travel up the stack, we can use a single global variable to store the multiple values with the function object that returned them. That way multiple-value-bind can check the array, and if it has values from any but the expected function object, we can just ignore them. To pass through multiple values, every tail call now has to check the global variable to see if multiple values got returned by the tail function. If so, it just replaces the function object they're tagged with with itself.
I can see that approach failing for a recursive function that sets multiple values in one of its sub-invocations, but doesn't return multiple values to whoever originally called it expecting multiple values.
Vladimir
Ok, I just pushed the promised change that removes mv passthrough and replaces the function object property setting with a global variable. I'm very happy with this approach - the generated code is now much cleaner than it used to be. Please try the changes out. If there aren't any issues, this is going in to the next release, which I'd like to make by the end of the month. Happy hacking, Vladimir
OK, I've tried out the latest commit. The changes to the MV implementation seem fine to me, though I haven't tested or analyzed them other than to confirm that our code isn't affected. I like that the generated code is simpler, and I agree with your choice to not support passthrough unless/until we can find a correct implementation that isn't too complicated. In the absence of that, the latest version seems like a good compromise and an improvement over what we had before. It's a reasonable place to stop. That being said, there's one point that's still bugging me because I don't see it yet:
Other exit points from a function are throws and control reaching the end of the function. So both are going to have to get instrumented.
I sketched an argument why extra instrumentation isn't needed in these two cases. If that argument is wrong (which it may well be) then we should be able to give examples where the pure global-variable (GV) design fails in the absence of such implementation. That is: (1) An example where an uninstrumented throw causes the GV implementation to be wrong; and (2) An example where an uninstrumented function exit causes the GV implementation to be wrong. I don't think any of the examples in the thread satisfy (1) or (2), nor have I come up with any. Can you? I'd like to see them. Daniel On Thu, Sep 13, 2012 at 12:00 AM, Vladimir Sedach <vsedach@gmail.com> wrote:
Ok, I just pushed the promised change that removes mv passthrough and replaces the function object property setting with a global variable. I'm very happy with this approach - the generated code is now much cleaner than it used to be.
Please try the changes out. If there aren't any issues, this is going in to the next release, which I'd like to make by the end of the month.
Happy hacking, Vladimir
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Hi Dan,
I sketched an argument why extra instrumentation isn't needed in these two cases. If that argument is wrong (which it may well be) then we should be able to give examples where the pure global-variable (GV) design fails in the absence of such implementation. That is:
(1) An example where an uninstrumented throw causes the GV implementation to be wrong; and
I thought that something like the following could be developed into a counterexample: (progn (defun foo (x) (try (if (= 0 x) (values 1 2 3) (bar)) (:catch (e) 27))) (defun bar () (foo 0) (throw 13)) (multiple-value-bind (a b c) (foo 1) (list a b c))) But because of implicit returns, I don't see how. I guess the implementation is safe in that respect. Vladimir
participants (3)
-
Daniel Gackle -
Red Daly -
Vladimir Sedach