[pro] Compiler APIs in Common Lisp
I am current reading an interesting paper on Racket's macro system (referenced here: http://lambda-the-ultimate.org/node/4196 ), and some of the objects they expose quack like CLTL-2 Environments: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html. I have read that having write access to environments makes it hard for the compiler to optimize code. However, even portable read only access would be helpful. Does anyone besides me and Cyrus Harmon (http://cyrusharmon.org/blog/display?id=113) miss Environments? Matt
On 12/04/2011, at 7:45 AM, Matthew Swank wrote:
I am current reading an interesting paper on Racket's macro system (referenced here: http://lambda-the-ultimate.org/node/4196 ), and some of the objects they expose quack like CLTL-2 Environments: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html. I have read that having write access to environments makes it hard for the compiler to optimize code. However, even portable read only access would be helpful. Does anyone besides me and Cyrus Harmon (http://cyrusharmon.org/blog/display?id=113) miss Environments?
Alas, I wish I had been exposed to them before. Until Cyrus' post I had no knowledge of them. I thank Cyrus for making that post. Using the function CLTL2 VARIABLE-INFORMATION function, I produced the following higher level functionality as a proof of concept for an idea I have for a mathematical optimisation library. (type-dispatching-defun matrix-add (y x scalar) (assert (= (array-total-size y) (array-total-size x))) (dotimes (index (array-total-size x)) (declare (type fixnum index)) (setf (row-major-aref y index) (+ (* scalar (row-major-aref x index)) (row-major-aref y index)))) y) (add-type-specific-function 'matrix-add '((simple-array single-float (*)) (simple-array single-float (*)) single-float)) (defun time-generic () (let ((a (make-array 1000 :initial-element 1)) (b (make-array 1000 :initial-element 2))) (time (dotimes (i 100000) (matrix-add a b 2.0))))) (defun time-single-float-array () (let ((a (make-array 1000 :initial-element 1.0 :element-type 'single-float)) (b (make-array 1000 :initial-element 2.0 :element-type 'single-float))) (declare (type (simple-array single-float (*)) a b)) (time (dotimes (i 100000) (matrix-add a b 2.0))))) TESTS> (time-generic) Evaluation took: 8.004 seconds of real time 8.000982 seconds of total run time (7.997856 user, 0.003126 system) 99.96% CPU 15,929,157,645 processor cycles 0 bytes consed TESTS> (time-single-float-array) Evaluation took: 0.430 seconds of real time 0.429092 seconds of total run time (0.428791 user, 0.000301 system) 99.77% CPU 854,425,193 processor cycles 0 bytes consed Given the possibilities it offers, I am interested to know why it isn't part of the standard. Thanks for the Racket links, I will have a read. Mark
Matt
_______________________________________________ pro mailing list pro@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/pro
On 12 Apr 2011, at 10:46, Mark Cox wrote:
On 12/04/2011, at 7:45 AM, Matthew Swank wrote:
I am current reading an interesting paper on Racket's macro system (referenced here: http://lambda-the-ultimate.org/node/4196 ), and some of the objects they expose quack like CLTL-2 Environments: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html. I have read that having write access to environments makes it hard for the compiler to optimize code. However, even portable read only access would be helpful. Does anyone besides me and Cyrus Harmon (http://cyrusharmon.org/blog/display?id=113) miss Environments?
Alas, I wish I had been exposed to them before. Until Cyrus' post I had no knowledge of them. I thank Cyrus for making that post. Using the function CLTL2 VARIABLE-INFORMATION function, I produced the following higher level functionality as a proof of concept for an idea I have for a mathematical optimisation library.
[...]
Given the possibilities it offers, I am interested to know why it isn't part of the standard.
There are several possible answers to this. Ideally, you would want a data structure that is based either on structs or on CLOS classes that gives you information about the lexical environment. However, the compiler must exist already at a very early phase of bootstrapping a Lisp system, and the more advanced types are probably not ready yet at that stage. Secondly, there are quite a few namespaces (not just 2) that all need to handled in some way in lexical environments. Ensuring a good API to inspect all of them was apparently not trivial. Thirdly, different compilers have different ways how to organize their compilation phases - making them all agree on how to provide the information is maybe also not trivial. And then, from a different angle, Common Lisp already provides environment objects and local macro definitions, and they already allow you to go a very long way. For example, I have found the discussion at http://www.lispworks.com/documentation/HyperSpec/Issues/iss066_w.htm very illuminating. (See the two local-type-declare examples - I particularly like the version using symbol macros.) I have implemented an experimental hygienic-compatible macro system for Common Lisp using just these ingredients. See http://dx.doi.org/10.3217/jucs-016-02-0271 for a description. I would strongly recommend considering using such techniques, instead of holding your breath and waiting for wide adoption of fully specified lexical environments. ;) (Just to avoid misunderstandings: such environments would definitely be more expressive than just opaque environment objects + local macros, it's just that environment objects + local macros are already a lot more powerful than some may expect!) Pascal -- Pascal Costanza The views expressed in this email are my own, and not those of my employer.
On 04/12/2011 01:29 PM, Pascal Costanza wrote:
And then, from a different angle, Common Lisp already provides environment objects and local macro definitions, and they already allow you to go a very long way. Yes, they do, but since the *macrolet forms aren't top level it's harder to get good ide (well, slime) support for debugging them.
For example, one technique I use when building a compile time language is to have all my context sensitive operators defined as top level macros that initially signal errors. Each operator gets redefined by macrolet when the expansion is in a context where the operation is valid. A simple example is here: http://paste.lisp.org/+2LPI You need macroexpand-all to get the proper expansion of a redefined (actually shadowed) operator. However, perhaps this is a quibble related to my use of stone tools for Lisp development :). Matt
Matthew Swank <akopa.gmane.poster@gmail.com> writes:
I am current reading an interesting paper on Racket's macro system (referenced here: http://lambda-the-ultimate.org/node/4196 ), and some of the objects they expose quack like CLTL-2 Environments: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html. I have read that having write access to environments makes it hard for the compiler to optimize code. However, even portable read only access would be helpful. Does anyone besides me and Cyrus Harmon (http://cyrusharmon.org/blog/display?id=113) miss Environments?
Matt
At least I have missed environments at some point. I think that Franz has put the most work in this direction, see http://www.franz.com/support/documentation/current/doc/environments.htm#envi... And if I remember posts by Duane Rettig on comp.lang.lisp right, they would be quite willing to cooperate on this issue with other implementors. Nicolas
On Mon, 11 Apr 2011 21:45:36 +0000 (UTC) Matthew Swank <akopa.gmane.poster@gmail.com> wrote:
I am current reading an interesting paper on Racket's macro system (referenced here: http://lambda-the-ultimate.org/node/4196 ), and some of the objects they expose quack like CLTL-2 Environments: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html. I have read that having write access to environments makes it hard for the compiler to optimize code. However, even portable read only access would be helpful. Does anyone besides me and Cyrus Harmon (http://cyrusharmon.org/blog/display?id=113) miss Environments?
I just wanted to say that the resulting thread is most precious and worthy of archiving; I was also occasionally confronted to various compilation-environment related issues and so far the provided answers and links seem to touch most, if not all of the topic: - Custom lexically scoped SETF/GETF expansions built on macros, using lexical macros overriding the top-level macros - Lexically scoped symbol-macrolet using lexically scoped macrolet - The ENV parameter of the various special forms, where available, and implementation-dependent related features - Custom declarations and implementation-dependent ways to access their current values in the current scope - The COMPILER-LET issue Great thanks, -- Matt
Matthew Mondor <mm_lists@...> writes:
I was also occasionally confronted to various compilation-environment related issues and so far the provided answers and links seem to touch most, if not all of the topic:
...
- Lexically scoped symbol-macrolet using lexically scoped macrolet - The ENV parameter of the various special forms, where available, and implementation-dependent related features ...
- The COMPILER-LET issue
I that spirit here is a primitive compiler-let (called compile-let-1) that uses some of those tools: http://paste.lisp.org/display/121480
participants (6)
-
Mark Cox -
Matthew D. Swank -
Matthew Mondor -
Matthew Swank -
Nicolas Neuss -
Pascal Costanza