[parenscript-devel] Feature request: prefixed names
Nowadays many things seem to want to include their JavaScript in your webpages. Aside from the fact that this is annoying, dangerous and wrong, we have to accept that it is quite prevalent: from Google AdWords to ReCaptcha. If I might add a feature request, it would be to be able to make sure that the identifiers generated by parenscript don't clash with anything else.
Hi John, That sounds like a good idea, but I think it would be difficult to implement correctly. The problem is that name mangling would have to take place both when you declare the variables/functions, and when you access them. So there would have to be some global state to keep track of which identifiers have already been declared in ParenScript code so that the corresponding name-mangling could take place when those identifiers are accessed. This presents problems with forward references - there's currently no notion of "whole Javascript program" in ParenScript, which besides wouldn't make sense if you're using ParenScript to write inline scripts. If everything is going to get name-mangled, then there would have to be some escape procedure for identifiers that you don't want to name mangle (functions in other Javascript libraries). Plus, you'd have to have a way to "export" identifiers when you want to make your own functions for other Javascript code to call. In short, I don't think there is a solution to this that would work well in more than one use case. Thanks, Vladimir On 6/19/07, John Fremlin <john@fremlin.org> wrote:
Nowadays many things seem to want to include their JavaScript in your webpages. Aside from the fact that this is annoying, dangerous and wrong, we have to accept that it is quite prevalent: from Google AdWords to ReCaptcha.
If I might add a feature request, it would be to be able to make sure that the identifiers generated by parenscript don't clash with anything else. _______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
On 6/20/07, Vladimir Sedach <vsedach@gmail.com> wrote:
Hi John,
...there's currently no notion of "whole Javascript program" in ParenScript, which besides wouldn't make sense if you're using ParenScript to write inline scripts. If everything is going to get name-mangled, then there would have to be some escape procedure for identifiers that you don't want to name mangle (functions in other Javascript libraries). Plus, you'd have to have a way to "export" identifiers when you want to make your own functions for other Javascript code to call. In short, I don't think there is a solution to this that would work well in more than one use case.
I prefer a Parenscript that departs from the exact semantics of Javascript and a Parenscript compiler that performs more advanced analysis. In this case, introducing the concept of a "whole Parenscript program" would be beneficial to Parenscript developers. It will not only allow a more sophisticated code generator (for minifying, prefixing, etc.) but could also make it easier to extend the compiler, generate documentation for a Parenscript program, and otherwise manipulate Parenscripts programmatically. I suspect that there are manageable solutions to problems like inconsistent "name manglings" across different compilations. A package system that keeps track of variable and function declarations can resolve identifier references and rewrite them accordingly, leaving global or unresolved identifiers alone. A few months ago I implemented the first steps of a package system for Parenscript that does some semantic analysis to accomplish prefixing as described.[1] I did not attempt to solve the "inline scripts" issue, but that problem seems far from impossible. Of course users have varying use cases. I write lengthy programs in standalone .paren files. Others may use Parenscript quite lightly to embed scripts in HTML. We can probably find solutions that make everyone's life easier. I am generally of the opinion that Parenscript should be as close to Lisp as possible. It is absurd that even in 2007 there is still so much fuss over programming languages. It would be great if a separate Parenscript language did not need to exist. -Red [1] remnants of that work are in the darcs repository for the Suave project in the /suavescript/ directory: http://common-lisp.net/project/suave/ Thanks,
Vladimir
On 6/19/07, John Fremlin <john@fremlin.org> wrote:
Nowadays many things seem to want to include their JavaScript in your webpages. Aside from the fact that this is annoying, dangerous and wrong, we have to accept that it is quite prevalent: from Google AdWords to ReCaptcha.
If I might add a feature request, it would be to be able to make sure that the identifiers generated by parenscript don't clash with anything else. _______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
"Red Daly" <reddaly@gmail.com> writes: [...]
I prefer a Parenscript that departs from the exact semantics of Javascript and a Parenscript compiler that performs more advanced analysis. In this case, introducing the concept of a "whole Parenscript program" would be beneficial to Parenscript developers. It will not only allow a more sophisticated code generator (for minifying, prefixing, etc.) but could also make it easier to extend the compiler, generate documentation for a Parenscript program, and otherwise manipulate Parenscripts programmatically.
Heartily agree.
I suspect that there are manageable solutions to problems like inconsistent "name manglings" across different compilations. A package system that keeps track of variable and function declarations can resolve identifier references and rewrite them accordingly, leaving global or unresolved identifiers alone. A few months ago I implemented the first steps of a package system for Parenscript that does some semantic analysis to accomplish prefixing as described.[1] I did not attempt to solve the "inline scripts" issue, but that problem seems far from impossible.
In fact, I don't see it as being a problem as most inline scripts are probably just calling functions already defined. If there were a sort of "parenscript program or package" class (PPP for now, until someone comes up with a proper name), then when you ask for parenscript to be generated, you have to pass in an instance of this class. Inline scripts would just mention which PPP they were related to so that name-mangling, function inlining(!) and consequent dead code elimination(!), global constant propagation(!), and so on could occur correctly. Another useful thing would be to be able to add lots of functions to the PPP and then have parenscript dump out only the ones that were actually referenced, avoiding the current practice of massive JavaScript library downloads. [...]
I am generally of the opinion that Parenscript should be as close to Lisp as possible. It is absurd that even in 2007 there is still so much fuss over programming languages. It would be great if a separate Parenscript language did not need to exist.
Hear, hear. In this case the business of the PPP dumping out just referenced functions would be quite handy, so that you wouldn't have to download the entire, inevitably massive, CL compatibility library for small functions. [...]
Ok, I can see this design working: The ParenScript compiler, instead of emitting a Javascript string, produces Lisp code to emit the Javascript string, where all identifier references are resolved when the code runs. So as you define new code in ParenScript, the necessary identifier information gets emitted along with the generated Javascript generator code, as code that updates that PPP in an eval-when clause of :compile-toplevel :load-toplevel :execute. Does that make sense? What does everyone think? This is going to take a lot of work, but I think the compiler could use some refactoring anyway. :) Vladimir On 6/21/07, John Fremlin <john@fremlin.org> wrote:
"Red Daly" <reddaly@gmail.com> writes: [...]
I prefer a Parenscript that departs from the exact semantics of Javascript and a Parenscript compiler that performs more advanced analysis. In this case, introducing the concept of a "whole Parenscript program" would be beneficial to Parenscript developers. It will not only allow a more sophisticated code generator (for minifying, prefixing, etc.) but could also make it easier to extend the compiler, generate documentation for a Parenscript program, and otherwise manipulate Parenscripts programmatically.
Heartily agree.
I suspect that there are manageable solutions to problems like inconsistent "name manglings" across different compilations. A package system that keeps track of variable and function declarations can resolve identifier references and rewrite them accordingly, leaving global or unresolved identifiers alone. A few months ago I implemented the first steps of a package system for Parenscript that does some semantic analysis to accomplish prefixing as described.[1] I did not attempt to solve the "inline scripts" issue, but that problem seems far from impossible.
In fact, I don't see it as being a problem as most inline scripts are probably just calling functions already defined.
If there were a sort of "parenscript program or package" class (PPP for now, until someone comes up with a proper name), then when you ask for parenscript to be generated, you have to pass in an instance of this class.
Inline scripts would just mention which PPP they were related to so that name-mangling, function inlining(!) and consequent dead code elimination(!), global constant propagation(!), and so on could occur correctly.
Another useful thing would be to be able to add lots of functions to the PPP and then have parenscript dump out only the ones that were actually referenced, avoiding the current practice of massive JavaScript library downloads.
[...]
I am generally of the opinion that Parenscript should be as close to Lisp as possible. It is absurd that even in 2007 there is still so much fuss over programming languages. It would be great if a separate Parenscript language did not need to exist.
Hear, hear.
In this case the business of the PPP dumping out just referenced functions would be quite handy, so that you wouldn't have to download the entire, inevitably massive, CL compatibility library for small functions.
[...]
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
Here is one that prefers that parenscript is kept much as it is, a simple sexp syntax form of javascript without lisp features. My ideal parenscript has these features: * A simple syntax. You can immediately know what the generated javascript code looks like. You can code without having to look at the generated code to trust that it is correct. * Complete and stable syntax, you can do everything that javascript does. * Easy to integrate with javascript code and libraries. * Well tested, supported by automatic tests of all features. This makes it easy to update the internal workings witout breaking user code. That is the reason I made the testcode that syncs with the documentation, I don't know how many use it though... * Decent performance. Thanks Attila for the latest speedup! I am very happy with parenscript as it is, as a high quality low level javascript translator close to my ideal. But I can agree that sometimes a higher level language would be nice, and I have also dreamt of all the features mentioned. I am willing to hack on a new language that is more high level, but then I suggest we have two layers: Parenscript as it is now (defined by my feature list above and the current syntax) as the low level language, call it parenscript-one. A new high level language (parenscript two) that compiles to parenscript one. But I suggest a clear division between the layers. Not everyone may want to use parenscript two for numerous reasons, but they can use and contribute to parenscript one. And some ideas by parenscript two people goes into parenscript one (such as compacting whitespace in the generated javascript). In fact, even Vladimirs idea to generate lisp code that generates javascript instead of direct strings might go into parenscript one, anything that doesn't change the syntax of the language. Lispers are a scarce resource and it is good if we can make all contribute to the same project. I think the current definition of parenscript as a direct sexp syntax of javascript is the lowest common denominator of everyones goals. But once you start to make something higher level peoples taste and needs will be more different. If you are tied to one of the big javascript frameworks you might be forced to use their implementation of objects. Or you might want to implement CLOS in javascript, as Red Daly does. Or have yet another homemade object implementation, as I currently have. Some want continuations, some don't care. We should do our best to make it possible to pick the features every need. But everyone needs a basic sexp syntax of javascript to build upon. Summary: Thumb up for a nice high level language, but keep a low level language as well. /Henrik Hjelte
"Henrik Hjelte" <henrik.hjelte@stix.to> writes: [...]
But I can agree that sometimes a higher level language would be nice, and I have also dreamt of all the features mentioned. I am willing to hack on a new language that is more high level, but then I suggest we have two layers: Parenscript as it is now (defined by my feature list above and the current syntax) as the low level language, call it parenscript-one. A new high level language (parenscript two) that compiles to parenscript one. But I suggest a clear division between the layers.
My idea was to try to make sure that any extra features added did not impact on usage of plain parenscript-one in any way (except perhaps to optimise the code generated without semantic changes), unless the extra features were actually used. It is a good point that this idea is probably quite optimistic in practice, and a clear way of being able to emit only parenscript-one should definitely be kept. [...]
"Henrik Hjelte" <henrik.hjelte@stix.to> writes: [...]
But I can agree that sometimes a higher level language would be nice, and I have also dreamt of all the features mentioned. I am willing to hack on a new language that is more high level, but then I suggest we have two layers: Parenscript as it is now (defined by my feature list above and the current syntax) as the low level language, call it parenscript-one. A new high level language (parenscript two) that compiles to parenscript one. But I suggest a clear division between the layers. --text follows this line-- My idea was to try to make sure that any extra features added did not impact on usage of plain parenscript-one in any way (except perhaps to optimise the code generated without semantic changes), unless the extra features were actually used.
It is a good point that this idea is probably quite optimistic in practice, and a clear way of being able to emit only parenscript-one should definitely be kept. [...]
"Vladimir Sedach" <vsedach@gmail.com> writes:
Ok, I can see this design working:
The ParenScript compiler, instead of emitting a Javascript string, produces Lisp code to emit the Javascript string, where all identifier references are resolved when the code runs. So as you define new code in ParenScript, the necessary identifier information gets emitted along with the generated Javascript generator code, as code that updates that PPP in an eval-when clause of :compile-toplevel :load-toplevel :execute.
Does that make sense? What does everyone think? This is going to take a lot of work, but I think the compiler could use some refactoring anyway. :)
I think the general use case is a sort of Parenscript library that is quite fixed and only uses identifiers in itself. The name references inside this library can presumably be all resolved at the host lisp's compile time. Then there are numerous small bits of JavaScript shoved into each page as event attributes or <script>s or whatever. It is unlikely that these things will modify the PPP (if they did, then you would have to have one PPP per page). But most of their references will probably be resolvable at compile time to identifiers in the library. I'm not sure that name-mangling (helpful by-product: spell-checking of identifiers) would really use the "lisp code templates to generate a string" functionality. However, for me, a compiler from parenscript to a sort of templatised lisp would be immensely useful. For example, I use a dynamic-hider thing, that makes a div with an onclick event: `(toggle-hiding ,other-html-element-id). If this could be compiled to a lisp template so that the string in other-html-element-id could be inserted directly into the output without calling the ParenScript compiler for each page generation, that would be great. It would make my web-application much, much faster. [...]
I think the general use case is a sort of Parenscript library that is quite fixed and only uses identifiers in itself. The name references inside this library can presumably be all resolved at the host lisp's compile time.
I think the important thing is to keep in mind that the time at which the ParenScript code gets "compiled" (however shallowly) into Lisp and the time at which that code gets compiled into Javascript don't necessarily have to be the same. This is what buys the name resolution in my proposed design relatively cheaply (from the standpoint of the cost of adding things to the ParenScript compiler).
Then there are numerous small bits of JavaScript shoved into each page as event attributes or <script>s or whatever. It is unlikely that these things will modify the PPP (if they did, then you would have to have one PPP per page).
Not necessarily. I was thinking you could have a form like (in-parenscript-package PPP) that would set some special variable indicating the PPP/namespace that the forthcoming ParenScript code would use for its identifiers. At compile time, the namespace would be populated and its binding done for name resolution, however the actual resolution of identifiers wouldn't take place until the JavaScript code was emitted.
I'm not sure that name-mangling (helpful by-product: spell-checking of identifiers) would really use the "lisp code templates to generate a string" functionality.
I think it would have to, otherwise how would forward references be resolved? Thanks, Vladimir
"Vladimir Sedach" <vsedach@gmail.com> writes: --text follows this line--
That sounds like a good idea, but I think it would be difficult to implement correctly. The problem is that name mangling would have to take place both when you declare the variables/functions, and when you access them. So there would have to be some global state to keep track of which identifiers have already been declared in ParenScript code so that the corresponding name-mangling could take place when those identifiers are accessed.
Yes, I am well aware. If it didn't involve so much hard work, I would have done it already. However, this is another motivation for parenscript moving more towards being a proper Lisp to JavaScript compiler. Eventually, being able to run the same code in Lisp on the server as in JavaScript on the client would obviously be awesome. The definition tracking functionality would also be useful for code-compression (choosing short identifiers) as someone else suggested. Of course, I suppose constant folding (as you suggested) is the first step and would definitely be much more practical. [...]
If you want to make sure that identifiers generated by Parenscript don't clash with those in third-party Javascript, wouldn't you have to parse the third-party Javascript? Daniel On 6/20/07, John Fremlin <john@fremlin.org> wrote:
"Vladimir Sedach" <vsedach@gmail.com> writes: --text follows this line--
That sounds like a good idea, but I think it would be difficult to implement correctly. The problem is that name mangling would have to take place both when you declare the variables/functions, and when you access them. So there would have to be some global state to keep track of which identifiers have already been declared in ParenScript code so that the corresponding name-mangling could take place when those identifiers are accessed.
Yes, I am well aware. If it didn't involve so much hard work, I would have done it already.
However, this is another motivation for parenscript moving more towards being a proper Lisp to JavaScript compiler. Eventually, being able to run the same code in Lisp on the server as in JavaScript on the client would obviously be awesome.
The definition tracking functionality would also be useful for code-compression (choosing short identifiers) as someone else suggested.
Of course, I suppose constant folding (as you suggested) is the first step and would definitely be much more practical.
[...]
_______________________________________________ parenscript-devel mailing list parenscript-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/parenscript-devel
"Daniel Gackle" <danielgackle@gmail.com> writes:
If you want to make sure that identifiers generated by Parenscript don't clash with those in third-party Javascript, wouldn't you have to parse the third-party Javascript?
In theory, yes. ;-) In practice just prefixing your identifiers with something unusual would achieve the same thing. [...]
participants (6)
-
Daniel Gackle -
Henrik Hjelte -
John Fremlin -
John Fremlin -
Red Daly -
Vladimir Sedach