I'm using ASDF to build an open source implementation of Google's protocol buffer data serialization library. The code is here:
http://github.com/brown/protobuf
I've defined a new component type that allows developers to include protobuf message definition files in their Lisp ASDF files:
(defclass protobuf-source-file (cl-source-file) ((relative-proto-pathname ... ) (search-path ... ) (:documentation "A protocol buffer definition file."))
When used like this in a system definition:
:components ((:protobuf-source-file "foo"))
executing the ASDF:LOAD-OP operation causes the file called foo.proto to be translated into foo.lisp by a C++ program, and then compiled into foo.fasl and loaded.
Note that foo.lisp is created in the same directory as the system definition, but I would prefer it to be created in the same directory as foo.fasl, which for me is in a subdirectory of ~/.cache/common-lisp.
What's the easiest way to accomplish this?
I initially tried overriding INPUT-FILES when applied to CL-SOURCE-FILE instances, but ASDF::PERFORM is defined like this:
(defmethod perform ((operation compile-op) (c cl-source-file)) (let ((source-file (component-pathname c)) ... ) ... )
Since it calls COMPONENT-PATHNAME to find Lisp source, it will always look in the system definition directory.
bob
On 8/5/10 Aug 5 -5:58 PM, Robert Brown wrote:
I'm using ASDF to build an open source implementation of Google's protocol buffer data serialization library. The code is here:
http://github.com/brown/protobuf
I've defined a new component type that allows developers to include protobuf message definition files in their Lisp ASDF files:
(defclass protobuf-source-file (cl-source-file) ((relative-proto-pathname ... ) (search-path ... ) (:documentation "A protocol buffer definition file."))
When used like this in a system definition:
:components ((:protobuf-source-file "foo"))
executing the ASDF:LOAD-OP operation causes the file called foo.proto to be translated into foo.lisp by a C++ program, and then compiled into foo.fasl and loaded.
Note that foo.lisp is created in the same directory as the system definition, but I would prefer it to be created in the same directory as foo.fasl, which for me is in a subdirectory of ~/.cache/common-lisp.
What's the easiest way to accomplish this?
I initially tried overriding INPUT-FILES when applied to CL-SOURCE-FILE instances, but ASDF::PERFORM is defined like this:
(defmethod perform ((operation compile-op) (c cl-source-file)) (let ((source-file (component-pathname c)) ... ) ... )
Since it calls COMPONENT-PATHNAME to find Lisp source, it will always look in the system definition directory.
Hm. Interesting. It looks like this reveals an uncertainty about the semantics of COMPONENT (and specifically COMPONENT-PATHNAME) versus INPUT-FILES.
LOAD-OP works on INPUT-FILES but COMPILE-OP does not, which seems like an unhappy departure from orthogonality....
This seems like another case where we understand the object model of ASDF (and the corresponding protocols) less well than might be desirable. Alas, I find no documentation for the intended meaning of INPUT-FILES (which probably accounts for certain difficulties in extending ASDF to new component classes and operations ;->).
best, r
On Thu, 2010-08-05 at 18:58 -0400, Robert Brown wrote:
I've defined a new component type that allows developers to include protobuf message definition files in their Lisp ASDF files:
(defclass protobuf-source-file (cl-source-file) ((relative-proto-pathname ... ) (search-path ... ) (:documentation "A protocol buffer definition file."))
When used like this in a system definition:
:components ((:protobuf-source-file "foo"))
executing the ASDF:LOAD-OP operation causes the file called foo.proto to be translated into foo.lisp by a C++ program, and then compiled into foo.fasl and loaded.
Note that foo.lisp is created in the same directory as the system definition, but I would prefer it to be created in the same directory as foo.fasl, which for me is in a subdirectory of ~/.cache/common-lisp.
What's the easiest way to accomplish this?
I initially tried overriding INPUT-FILES when applied to CL-SOURCE-FILE instances, but ASDF::PERFORM is defined like this:
(defmethod perform ((operation compile-op) (c cl-source-file)) (let ((source-file (component-pathname c)) ... ) ... )
Since it calls COMPONENT-PATHNAME to find Lisp source, it will always look in the system definition directory.
Try something like this:
(defmethod asdf:perform ((op asdf:compile-op) (c protobuf-source-file)) (let* ((output-defaults (pathname (car (asdf:output-files op c)))) (generated-source-file (make-pathname :name (pathname-name (asdf:component-pathname c)) :type "lisp" :defaults output-defaults))) ))
You need to call OUTPUT-FILES which will apply the output translations you've configured
On Fri, Aug 6, 2010 at 1:24 AM, Stelian Ionescu sionescu@cddr.org wrote:
Try something like this:
(defmethod asdf:perform ((op asdf:compile-op) (c protobuf-source-file)) (let* ((output-defaults (pathname (car (asdf:output-files op c)))) (generated-source-file (make-pathname :name (pathname-name (asdf:component-pathname c)) :type "lisp" :defaults output-defaults))) ))
You need to call OUTPUT-FILES which will apply the output translations you've configured
Your suggestion does work, but it's a bit mysterious. Note that you didn't redefine INPUT-FILES for components of type PROTOBUF-SOURCE-FILE and the suggestion involves copying PERFORM code from asdf.lisp into my protobuf.asd file.
Also, your change appears to just affect compilation, but due to ASDF's handling of so called "self dependencies" (at least I think this is the cause) your change also magically makes LOAD-OP function correctly.
I agree with Robert Goldman that it would be nice to pin down the semantics of INPUT-FILES and OUTPUT-FILES. Maybe the self-dependency code can be eliminated or made more general. Right now I think it's a special case just used to make LOAD-OP capable of doing two things, compiling and then loading the compiled products.
In any case, thanks for the suggestion. It does indeed solve my problem.
bob
On 8/6/10 Aug 6 -11:49 AM, Robert Brown wrote:
On Fri, Aug 6, 2010 at 1:24 AM, Stelian Ionescu <sionescu@cddr.org mailto:sionescu@cddr.org> wrote:
Try something like this: (defmethod asdf:perform ((op asdf:compile-op) (c protobuf-source-file)) (let* ((output-defaults (pathname (car (asdf:output-files op c)))) (generated-source-file (make-pathname :name (pathname-name (asdf:component-pathname c)) :type "lisp" :defaults output-defaults))) )) You need to call OUTPUT-FILES which will apply the output translations you've configured
This example suggests a possible improvement to the API to avoid problems down the road.
I'd suggest we add a new exported function, ASDF:OUTPUT-FILE (singular), which has the same signature as ASDF:OUTPUT-FILES.
ASDF:OUTPUT-FILE should return the first element of the list returned by ASDF:OUTPUT-FILES, if this list is of length 1. If ASDF:OUTPUT-FILES returns a list of length != 1, ASDF:OUTPUT-FILE should raise an error.
This is for people who are writing some code that is correct under the assumption that there will be only a single output-file. I bet a lot of people call (first (asdf:output-files ...)), effectively just hoping that no one went around behind their backs and changed the number of output files.
best, r
-
Robert Brown -
Robert Goldman -
Stelian Ionescu