[parenscript-devel] Keyword arguments and quoted symbols in Parenscript
Hello ParenScripters, I have modified the behavior of keywords found in Parenscript source to be parsed into (PS-QUOTE keyword) instead of (JS-VARIABLE keyword). This seems to make more sense because in Lisp, evaluating a keyword yields the keyword itself rather than the value bound to it. As a result, the ability to paass keyword arguments to functions is now restored. CL-USER> (ps:ps (foo "bar" :quix "quo" :etc "etc...")) "foo('bar', { quix : 'quo', etc : 'etc...' });" Whereas this used to yield foo('bar', quix, 'quo', etc, 'etc...') In general, symbols occupy a strange place in Parenscript. Javascript has no analogue of Lisp symbols, so there is no sensible translation of symbols that works for everyone. Nonetheless, quoted symbols are used in many Parenscript macros to fake having real symbols in javascript. We could build in the ability to translate symbols to some javascript form, e.g. (ps:ps (make-instance 'animal)) => makeInstance( symbolTable["ANIMAL"] ) The latter part of this post is just food for thought. I will go ahead and commit the patch for the former part of this post unless someone objects. Red
If I understand correctly, then yes, we object. We have a lot of code that uses keywords as standalone values (e.g. tokens in an AST), not keyword args to functions. It's important to have a way of treating keywords that allows for both of these usages. In other words, a keyword's presence in a lambda list might be (A) the name of a keyword arg or (B) a regular old function arg (a runtime value). The trouble with the old implementation of keyword args in PS is that it forced interpretation (A) on all keywords in lambda lists, breaking a good many of our functions. The code that's in PS right now supports both usages. It relies on the JS "arguments" keyword to allow functions with keyword args to pick out the dynamic portion of their lambda list without requiring them to be bundled into a magic JSON object under-the-hood. That magic JSON object caused us some trouble as well. We have a layer of our system written in a subset of CL that we are able to compile into PS using macros. (This has critical business value as it allows us to do some things which our competitors cannot.) The magic JSON object created an impedance mismatch with Lisp that made it impossible for us to write code in this subset that used keyword args (we'd have to write a separate JS version to suck function args out of the JSON object, which has no counterpart on the Lisp side). In general, although I'm skeptical of attempts to make PS overly Lispy in ways that don't have good equivalents in JS semantics, I also think that where there are opportunities to align Lisp and PS semantics naturally, we should do so. Daniel On Sat, Apr 25, 2009 at 9:38 AM, Red Daly <reddaly@gmail.com> wrote:
Hello ParenScripters,
I have modified the behavior of keywords found in Parenscript source to be parsed into (PS-QUOTE keyword) instead of (JS-VARIABLE keyword). This seems to make more sense because in Lisp, evaluating a keyword yields the keyword itself rather than the value bound to it. As a result, the ability to paass keyword arguments to functions is now restored.
CL-USER> (ps:ps (foo "bar" :quix "quo" :etc "etc...")) "foo('bar', { quix : 'quo', etc : 'etc...' });"
Whereas this used to yield foo('bar', quix, 'quo', etc, 'etc...')
In general, symbols occupy a strange place in Parenscript. Javascript has no analogue of Lisp symbols, so there is no sensible translation of symbols that works for everyone. Nonetheless, quoted symbols are used in many Parenscript macros to fake having real symbols in javascript. We could build in the ability to translate symbols to some javascript form, e.g.
(ps:ps (make-instance 'animal)) => makeInstance( symbolTable["ANIMAL"] )
The latter part of this post is just food for thought. I will go ahead and commit the patch for the former part of this post unless someone objects.
Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
On Sat, Apr 25, 2009 at 12:02 PM, Daniel Gackle <danielgackle@gmail.com>wrote:
If I understand correctly, then yes, we object.
We have a lot of code that uses keywords as standalone values (e.g. tokens in an AST), not keyword args to functions.
It's important to have a way of treating keywords that allows for both of
these usages. In other words, a keyword's presence in a lambda list might be (A) the name of a keyword arg or (B) a regular old function arg (a runtime value).
What do you mean by "a runtime value?" In the current Parenscript keywords are compiled to the variable with the same name as the symbol. If you wanted a variable reference, you could replace a form like (foo :bar whiz) to (foo bar whiz).
The trouble with the old implementation of keyword args in PS is that it forced interpretation (A) on all keywords in lambda lists, breaking a good many of our functions.
I am still confused about what functions this assumption is breaking, since quoted symbols are not translated into a JS equivalent. Can you provide a concrete example?
The code that's in PS right now supports both usages. It relies on the JS "arguments" keyword to allow functions with keyword args to pick out the dynamic portion of their lambda list without requiring them to be bundled into a magic JSON object under-the-hood. That magic JSON object caused us some trouble as well.
The magic JSON object should be transparent to you when you define keyword arguments. That is, unless you are talking about a &rest or &keyword-object, both of which operate differently for parenscript functions than lisp functions.
We have a layer of our system written in a subset of CL that we are able to compile into PS using macros. (This has critical business value as it allows us to do some things which our competitors cannot.)
This is a slick feature of Parenscript. Hopefully at some point we will see a full lisp -> javascript converter so that more code can be shared between client/server. Parenscript already does a lot more than any other solution out there for client/server code compatability (besides writing the server side in Javascript).
The magic JSON object created an impedance mismatch with Lisp that made it impossible for us to write code in this subset that used keyword args (we'd have to write a separate JS version to suck function args out of the JSON object, which has no counterpart on the Lisp side). In general, although I'm skeptical of attempts to make PS overly Lispy in ways that don't have good equivalents in JS semantics, I also think that where there are opportunities to align Lisp and PS semantics naturally, we should do so.
Daniel Agreed, for Parenscript. I would prefer true Lisp that compiles to Javascript, but I don't think that's what Parenscript should be. -Red
On Sat, Apr 25, 2009 at 9:38 AM, Red Daly <reddaly@gmail.com> wrote:
Hello ParenScripters,
I have modified the behavior of keywords found in Parenscript source to be parsed into (PS-QUOTE keyword) instead of (JS-VARIABLE keyword). This seems to make more sense because in Lisp, evaluating a keyword yields the keyword itself rather than the value bound to it. As a result, the ability to paass keyword arguments to functions is now restored.
CL-USER> (ps:ps (foo "bar" :quix "quo" :etc "etc...")) "foo('bar', { quix : 'quo', etc : 'etc...' });"
Whereas this used to yield foo('bar', quix, 'quo', etc, 'etc...')
In general, symbols occupy a strange place in Parenscript. Javascript has no analogue of Lisp symbols, so there is no sensible translation of symbols that works for everyone. Nonetheless, quoted symbols are used in many Parenscript macros to fake having real symbols in javascript. We could build in the ability to translate symbols to some javascript form, e.g.
(ps:ps (make-instance 'animal)) => makeInstance( symbolTable["ANIMAL"] )
The latter part of this post is just food for thought. I will go ahead and commit the patch for the former part of this post unless someone objects.
Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Hi Red,
I have modified the behavior of keywords found in Parenscript source to be parsed into (PS-QUOTE keyword) instead of (JS-VARIABLE keyword).
The current behavior is (ps::compile-parenscript-form :baz) => :baz (there's no quoting involved). The printer then outputs keyword symbols as strings, since this is the closest analogue (a self-evaluating object) to keywords among the native JavaScript objects.
This seems to make more sense because in Lisp, evaluating a keyword yields the keyword itself rather than the value bound to it. As a result, the ability to paass keyword arguments to functions is now restored.
CL-USER> (ps:ps (foo "bar" :quix "quo" :etc "etc...")) "foo('bar', { quix : 'quo', etc : 'etc...' });"
Whereas this used to yield foo('bar', quix, 'quo', etc, 'etc...')
This is actually a fix to keyword argument handling: previously the PS lambda-list handling code assumed that whenever it encountered a keyword in the arglist of a function, it was the beginning of the keyword portion of the lambda list, and started making the object at that point. This of course may not be the case if you are trying to use keywords as arguments to a function. With keywords compiled to strings, you get self-evaluating objects and can do keyword handling in the function body at runtime (which is what the current code does). Thanks, Vladimir
In general, symbols occupy a strange place in Parenscript. Javascript has no analogue of Lisp symbols, so there is no sensible translation of symbols that works for everyone. Nonetheless, quoted symbols are used in many Parenscript macros to fake having real symbols in javascript. We could build in the ability to translate symbols to some javascript form, e.g.
(ps:ps (make-instance 'animal)) => makeInstance( symbolTable["ANIMAL"] )
The latter part of this post is just food for thought. I will go ahead and commit the patch for the former part of this post unless someone objects.
Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
On Sun, Apr 26, 2009 at 8:03 AM, Vladimir Sedach <vsedach@gmail.com> wrote:
Hi Red,
I have modified the behavior of keywords found in Parenscript source to be parsed into (PS-QUOTE keyword) instead of (JS-VARIABLE keyword).
The current behavior is (ps::compile-parenscript-form :baz) => :baz (there's no quoting involved). The printer then outputs keyword symbols as strings, since this is the closest analogue (a self-evaluating object) to keywords among the native JavaScript objects.
This seems to make more sense because in Lisp, evaluating a keyword yields the keyword itself rather than the value bound to it. As a result, the ability to
It appears that I had an old version. I was using the darcs repo instead of the git repo. There must have been a lot of changes as of August! (ps:ps :foo) used to output a variable reference to foo. Nonetheless, here is a bug with the current implementation: CL-USER> (setf (ps:PS-PACKAGE-PREFIX :cl-user) "CL_USER_") CL-USER> (ps:ps (let* ((some-prop 'foo-sym) (my-object (ps:create 'foo-sym "value!"))) (slot-value my-object 'foo-sym))) "var CL_USER_someProp = 'foo-sym'; var CL_USER_myObject = { 'foo-sym' : 'value!' }; CL_USER_myObject.CL_USER_fooSym;" The expected behavior would be for the (create ...) form to reference the same value as the (slot-value ...) form. If there were any sort of Parenscript runtime, it would be easy to make a data structure for symbols. For example, the above code could translate to something like... function Symbol(string, prefix) { this.string = string; this.prefix = prefix; // .. add to symbol table, etc. } Symbol.prototype.toString = function() { if (this.prefix) return this.prefix + "::" + this.string; else return this.string; } var someProp_symbol = new Symbol("someProp", "CL_USER"); var CL_USER_someProp = someProp_symbol; var CL_USER_myObject = { someProp_symbol : 'value!' }; CL_USER_myObject[someProp_symbol]; This would differentiate symbols and strings, and it would result in pointer comparisons between symbols, just as in lisp. I understand the current approach is to avoid any runtime, but the runtime solution does seem like less of a hack in the end. paass
keyword arguments to functions is now restored.
CL-USER> (ps:ps (foo "bar" :quix "quo" :etc "etc...")) "foo('bar', { quix : 'quo', etc : 'etc...' });"
Whereas this used to yield foo('bar', quix, 'quo', etc, 'etc...')
This is actually a fix to keyword argument handling: previously the PS
lambda-list handling code assumed that whenever it encountered a keyword in the arglist of a function, it was the beginning of the keyword portion of the lambda list, and started making the object at that point. This of course may not be the case if you are trying to use keywords as arguments to a function. With keywords compiled to strings, you get self-evaluating objects and can do keyword handling in the function body at runtime (which is what the current code does).
I see the latest implementation now processes keyword arguments differently, eliminating the old hash table approach. This seems like an okay solution, but I suspect it adds a a little overhead for each call to a function with keyword arguments. I also noticed a bug that resulted in improper handling of the following case: (ps:ps (defun hello-world (&key ((my-name-key my-name))) (print "hello, " + my-name)) (hello-world 'my-name-key "fred")) "function CL_USER_helloWorld() { var CL_USER_myName; var _js54 = arguments.length; for (var n52 = 0; n52 < _js54; n52 += 2) { switch (arguments[n52]) { case CL_USER_myNameKey: { CL_USER_myName = arguments[n52 + 1]; }; }; }; if (CL_USER_myName === undefined) { CL_USER_myName = null; }; print('hello, ', plus, CL_USER_myName); }; CL_USER_helloWorld('my-name-key', 'fred');" the javascript should really use the following case: case 'myNameKey' instead of case CL_USER_myNameKey. a patch is attached All the best, Red
Thanks, Vladimir
In general, symbols occupy a strange place in Parenscript. Javascript has no analogue of Lisp symbols, so there is no sensible translation of symbols that works for everyone. Nonetheless, quoted symbols are used in many Parenscript macros to fake having real symbols in javascript. We could build in the ability to translate symbols to some javascript form, e.g.
(ps:ps (make-instance 'animal)) => makeInstance( symbolTable["ANIMAL"] )
The latter part of this post is just food for thought. I will go ahead and commit the patch for the former part of this post unless someone objects.
Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Hi Red,
It appears that I had an old version. I was using the darcs repo instead of the git repo. There must have been a lot of changes as of August! (ps:ps :foo) used to output a variable reference to foo.
Yes! All of them should be for the better, but if you see something that looks wrong, do let us know.
Nonetheless, here is a bug with the current implementation:
CL-USER> (setf (ps:PS-PACKAGE-PREFIX :cl-user) "CL_USER_") CL-USER> (ps:ps (let* ((some-prop 'foo-sym) (my-object (ps:create 'foo-sym "value!"))) (slot-value my-object 'foo-sym)))
"var CL_USER_someProp = 'foo-sym'; var CL_USER_myObject = { 'foo-sym' : 'value!' }; CL_USER_myObject.CL_USER_fooSym;"
The expected behavior would be for the (create ...) form to reference the same value as the (slot-value ...) form.
Just pushed a patch for that, which makes quoted symbol handling uniform.
If there were any sort of Parenscript runtime, it would be easy to make a data structure for symbols. For example, the above code could translate to something like...
function Symbol(string, prefix) { this.string = string; this.prefix = prefix; // .. add to symbol table, etc. } Symbol.prototype.toString = function() { if (this.prefix) return this.prefix + "::" + this.string; else return this.string; } var someProp_symbol = new Symbol("someProp", "CL_USER");
var CL_USER_someProp = someProp_symbol; var CL_USER_myObject = { someProp_symbol : 'value!' }; CL_USER_myObject[someProp_symbol];
This would differentiate symbols and strings, and it would result in pointer comparisons between symbols, just as in lisp.
I understand the current approach is to avoid any runtime, but the runtime solution does seem like less of a hack in the end.
I think there should be a runtime system that PS users would have the option of using for their applications, which would have more run-time CL features like first-class symbols. That way we can make Parenscript self-hosting, among other cool things.
I see the latest implementation now processes keyword arguments differently, eliminating the old hash table approach. This seems like an okay solution, but I suspect it adds a a little overhead for each call to a function with keyword arguments.
This approach adds overhead that is linear in the number of keyword arguments defined, while the previous approach added overhead that was linear in the number of times the keyword arguments were referenced (because it did a slot reference on each occurrence of the variable). I suspect a larger loss will result from the fact that the new approach uses the arguments array, which might prevent the JS compiler from making some kinds of optimizations, but without benchmarks all of this is just speculation.
I also noticed a bug that resulted in improper handling of the following case: (ps:ps (defun hello-world (&key ((my-name-key my-name))) (print "hello, " + my-name)) (hello-world 'my-name-key "fred"))
"function CL_USER_helloWorld() { var CL_USER_myName; var _js54 = arguments.length; for (var n52 = 0; n52 < _js54; n52 += 2) { switch (arguments[n52]) { case CL_USER_myNameKey:
{ CL_USER_myName = arguments[n52 + 1];
};
};
}; if (CL_USER_myName === undefined) { CL_USER_myName = null;
}; print('hello, ', plus, CL_USER_myName); }; CL_USER_helloWorld('my-name-key', 'fred');"
the javascript should really use the following case: case 'myNameKey' instead of case CL_USER_myNameKey. a patch is attached
Pushed. Thanks a lot for the patch! Vladimir
All the best, Red
Thanks, Vladimir
In general, symbols occupy a strange place in Parenscript. Javascript has no analogue of Lisp symbols, so there is no sensible translation of symbols that works for everyone. Nonetheless, quoted symbols are used in many Parenscript macros to fake having real symbols in javascript. We could build in the ability to translate symbols to some javascript form, e.g.
(ps:ps (make-instance 'animal)) => makeInstance( symbolTable["ANIMAL"] )
The latter part of this post is just food for thought. I will go ahead and commit the patch for the former part of this post unless someone objects.
Red
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
participants (3)
-
Daniel Gackle -
Red Daly -
Vladimir Sedach