Hi, I am exploring the possibility of using ASDF to define systems of non-Lisp code, specifically for Maxima (http://maxima.sourceforge.net). Maxima is written in Lisp but has its own language. I'd like to be able to load programs written in Maxima's language and/or Lisp.
It seems like ASDF should be suitable if I can just replace calls to COMPILE-FILE with Maxima's compile_file, and replace LOAD with Maxima's load. After looking over the 3.1.5 source code, it seems like one could clobber COMPILE-FILE* and LOAD* with Maxima-specific versions. That seems suboptimal, since it would be helpful to have the original versions of those functions for loading Lisp files.
Can someone give some advice about how to go about adapting ASDF to Maxima? Have there been similar attempts for other applications?
The bigger picture is that I'd like to use Quicklisp to download and install Maxima packages, and Quicklisp uses ASDF to compile and load stuff.
Thanks in advance for any light you can shed on this problem.
Robert Dodier
On 9/17/15 Sep 17 -12:57 PM, Robert Dodier wrote:
Hi, I am exploring the possibility of using ASDF to define systems of non-Lisp code, specifically for Maxima (http://maxima.sourceforge.net). Maxima is written in Lisp but has its own language. I'd like to be able to load programs written in Maxima's language and/or Lisp.
It seems like ASDF should be suitable if I can just replace calls to COMPILE-FILE with Maxima's compile_file, and replace LOAD with Maxima's load. After looking over the 3.1.5 source code, it seems like one could clobber COMPILE-FILE* and LOAD* with Maxima-specific versions. That seems suboptimal, since it would be helpful to have the original versions of those functions for loading Lisp files.
Can someone give some advice about how to go about adapting ASDF to Maxima? Have there been similar attempts for other applications?
The bigger picture is that I'd like to use Quicklisp to download and install Maxima packages, and Quicklisp uses ASDF to compile and load stuff.
This should be easy for you to do.
First, you need to subclass some kind of component for maxima source files.
You could either subclass cl-source-file, or perhaps something more foundational.
You should, at least, provide methods for
PERFORM LOAD-OP <YOUR-CLASS> and PERFORM COMPILE-OP <YOUR-CLASS>
I believe information about doing this should be in the ASDF manual.
You may also need to tell ASDF what the file-extension of your types of file is.
If there is a MAXIMA equivalent of a FASL file, you need to define methods for OUTPUT-FILES and INPUT-FILES for your operations on your component class.
That should be most of the solution.
Then you can make a new subclass of SYSTEM that will establish your component type as the default file type for that system. Then if you specify that you are making one of your system subclass type (see the manual for how to do this), then every :FILE component will be a MAXIMA-SOURCE-FILE instead of a CL-SOURCE-FILE.
You might also want to add an implicit DEPENDS-ON to MAXIMA for this system type.
I think that should give you what you need to find more info in the manual.
If there's something missing from the manual that you find you need to do, please post a ticket to ASDF on launchpad.net so that we can make necessary improvements to the manual.
Best, r
Take a look at https://github.com/brown/protobuf/blob/master/protobuf.asd and the other ".asd" files in that project.
The file above modifies ASDF so that it can compile and load Google's protocol buffer definition files. It may be more complicated than you need because ".proto" files are first translated into ".lisp" files, then compiled or loaded.
Bob
On Thu, Sep 17, 2015 at 1:57 PM, Robert Dodier robert.dodier@gmail.com wrote:
Hi, I am exploring the possibility of using ASDF to define systems of non-Lisp code, specifically for Maxima (http://maxima.sourceforge.net). Maxima is written in Lisp but has its own language. I'd like to be able to load programs written in Maxima's language and/or Lisp.
It seems like ASDF should be suitable if I can just replace calls to COMPILE-FILE with Maxima's compile_file, and replace LOAD with Maxima's load. After looking over the 3.1.5 source code, it seems like one could clobber COMPILE-FILE* and LOAD* with Maxima-specific versions. That seems suboptimal, since it would be helpful to have the original versions of those functions for loading Lisp files.
Can someone give some advice about how to go about adapting ASDF to Maxima? Have there been similar attempts for other applications?
The bigger picture is that I'd like to use Quicklisp to download and install Maxima packages, and Quicklisp uses ASDF to compile and load stuff.
Thanks in advance for any light you can shed on this problem.
Robert Dodier
This section of the manual, and the surrounding chapter, may help: https://common-lisp.net/project/asdf/asdf/Creating-new-operations.html#Creat...
You basically need to define a new component :maxima-file, :maxima, or whatever the name.
If it produces a Lisp file that then gets compiled to FASL, make it a subclass of cl-source-file indeed, and add a process-op or such to create the lisp file from the maxima source (see how cffi-grovel:grovel-file does it).
Otherwise, make it a subclass of source-file (or then again, possibly cl-source-file, if it produces FASLs), and define methods for compile-op and load-op on it.
Do NOT for whatever reason clobber any existing function (unless they have bugs indeed, in which case, submit a patch) — there lies madness (I should know, we were doing that at ITA before I bit the bullet and became ASDF maintainer). Define new classes and use ad hoc polymorphism as designed to specialize the behavior of the software.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Guns & bullets don't kill people — blood loss and organ damage kills people.
Thanks a lot to everybody for the advice. I think I've made some progress. I've attached a patch showing the initial changes I've made, and with this much I can load a trivial system into Maxima.
I'm working with asdf.lisp 2.26 which is packaged with Quicklisp, since the larger goal is to use Quicklisp to download and install Maxima packages.
Incidentally I'm working with Clisp 2.49.
I've run into some strangeness, maybe someone can explain what's going on.
(1) (asdf:load-system :foo) seems to always cause files to be compiled. That's kind of puzzling.
(2) The first time asdf.lisp is loaded, Quicklisp writes asdf.fas into its own cache. The second time, I get an error about ASDF:OOS. Any ideas? Here's the error:
$ maxima ;; Loading file /home/robert/.clisprc.lisp ... ;; Loading file /home/robert/quicklisp/setup.lisp ... HELLO FROM QUICKLISP/ASDF.LISP !!
*** - EVAL: undefined function ASDF:OOS The following restarts are available: USE-VALUE :R1 Input a value to be used instead of (FDEFINITION 'ASDF:OOS). RETRY :R2 Retry STORE-VALUE :R3 Input a new value for (FDEFINITION 'ASDF:OOS). SKIP :R4 skip (LET # #) RETRY :R5 retry (LET # #) STOP :R6 stop loading file /home/robert/quicklisp/setup.lisp SKIP :R7 skip (LET # #) RETRY :R8 retry (LET # #) STOP :R9 stop loading file /home/robert/.clisprc.lisp Break 1 QL-SETUP[4]>
Thanks again,
Robert
PS. Here's the example foo.asd I've been working with.
(defsystem foo :name "foo" :version "2015.09.21" :maintainer "Robert Dodier" :author "Robert Dodier" :licence "GPL" :description "Foo, Bar, and Baz" :long-description "Apply Foo's algorithm to Bar, yielding Baz"
:components ((:maxima-source-file "foo1") (:maxima-source-file "foo2")))
On 9/21/15 Sep 21 -6:15 PM, Robert Dodier wrote:
Thanks a lot to everybody for the advice. I think I've made some progress. I've attached a patch showing the initial changes I've made, and with this much I can load a trivial system into Maxima.
I'm working with asdf.lisp 2.26 which is packaged with Quicklisp, since the larger goal is to use Quicklisp to download and install Maxima packages.
Incidentally I'm working with Clisp 2.49.
I've run into some strangeness, maybe someone can explain what's going on.
(1) (asdf:load-system :foo) seems to always cause files to be compiled. That's kind of puzzling.
I don't know if this is why, but I see that there are no dependencies in your system. Was that intentional? Did you expect DEFSYSTEM to default to :SERIAL T.
If that causes this problem, though, that would be an ASDF bug.
(2) The first time asdf.lisp is loaded, Quicklisp writes asdf.fas into its own cache. The second time, I get an error about ASDF:OOS. Any ideas? Here's the error:
This isn't enough for me to help, since it looks like "maxima" is some kind of script that does something. I don't know what. Also, I don't know what's in your .clisprc.lisp...
At the bare minimum, we'd need to see a backtrace from the debugger.
cheers, r
$ maxima ;; Loading file /home/robert/.clisprc.lisp ... ;; Loading file /home/robert/quicklisp/setup.lisp ... HELLO FROM QUICKLISP/ASDF.LISP !!
*** - EVAL: undefined function ASDF:OOS The following restarts are available: USE-VALUE :R1 Input a value to be used instead of (FDEFINITION 'ASDF:OOS). RETRY :R2 Retry STORE-VALUE :R3 Input a new value for (FDEFINITION 'ASDF:OOS). SKIP :R4 skip (LET # #) RETRY :R5 retry (LET # #) STOP :R6 stop loading file /home/robert/quicklisp/setup.lisp SKIP :R7 skip (LET # #) RETRY :R8 retry (LET # #) STOP :R9 stop loading file /home/robert/.clisprc.lisp Break 1 QL-SETUP[4]>
Thanks again,
Robert
PS. Here's the example foo.asd I've been working with.
(defsystem foo :name "foo" :version "2015.09.21" :maintainer "Robert Dodier" :author "Robert Dodier" :licence "GPL" :description "Foo, Bar, and Baz" :long-description "Apply Foo's algorithm to Bar, yielding Baz"
:components ((:maxima-source-file "foo1") (:maxima-source-file "foo2")))
On Mon, Sep 21, 2015 at 7:15 PM, Robert Dodier robert.dodier@gmail.com wrote:
Thanks a lot to everybody for the advice. I think I've made some progress. I've attached a patch showing the initial changes I've made, and with this much I can load a trivial system into Maxima.
Do NOT patch ASDF, unless there's a bug. Create an extension and use defsystem-depends-on, as documented in the manual.
I'm working with asdf.lisp 2.26 which is packaged with Quicklisp, since the larger goal is to use Quicklisp to download and install Maxima packages.
Do NOT use ASDF 2.26. It is not supported, and internals are somewhat incompatible. ASDF 3 is available on every single maintained implementation.
Incidentally I'm working with Clisp 2.49.
CLISP isn't currently maintained. Unless you need it due for portability reasons, or love it enough to become its new maintainer, I suggest to use another implementation. If you *must* use CLISP, use our script tools/install-asdf.lisp to overwrite its ASDF with the latest ASDF release (currently 3.1.5).
I've run into some strangeness, maybe someone can explain what's going on.
(1) (asdf:load-system :foo) seems to always cause files to be compiled. That's kind of puzzling.
This typically means that your input-files or output-files are wrong. Your code is not even wrong: it's missing a method on output-files. ASDF 2.26 has also deep unfixable bugs regarding incremental build, and it is unsupported. Use ASDF 3.
(2) The first time asdf.lisp is loaded, Quicklisp writes asdf.fas into its own cache. The second time, I get an error about ASDF:OOS. Any ideas? Here's the error:
Use ASDF 3.
*** - EVAL: undefined function ASDF:OOS The following restarts are available: USE-VALUE :R1 Input a value to be used instead of (FDEFINITION 'ASDF:OOS). RETRY :R2 Retry STORE-VALUE :R3 Input a new value for (FDEFINITION 'ASDF:OOS). SKIP :R4 skip (LET # #) RETRY :R5 retry (LET # #) STOP :R6 stop loading file /home/robert/quicklisp/setup.lisp SKIP :R7 skip (LET # #) RETRY :R8 retry (LET # #) STOP :R9 stop loading file /home/robert/.clisprc.lisp Break 1 QL-SETUP[4]>
This is completely useless. Give us a backtrace. From CCL or SBCL, not CLISP. While compiling ASDF and/or your extension with extra debugging information.
Also, don't use (component-pathname c) in your perform method. Use (first (input-files o c)), so users can suitable modify or extend your methods.
Please come back after you've read the manual (notably section 7) and actually tried to apply the recipes there. And/or at least cargo culted from an existing extension.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org A politician divides mankind into two classes: tools and enemies. — Nietzsche
Hey everybody,
I've made some more progress with a Maxima extension for ASDF (attached). At this point it works pretty much as expected, for the simple examples I've tried. I think you should be able to use in Maxima like this: load("maxima_asdf.lisp"); asdf_load("foo"); to load foo.asd, which may contain :maxima-file components. asdf_load translates :maxima-files and compiles the resulting Lisp, while asdf_load_source just interprets the :maxima-files.
At this point the only thing I wish I could change (and this is a minor item) is that output file paths are something like $HOME/.cache/common-lisp/<lisp-version>/<path-to-source-file>/<maxima-version>/<system-name>/<file-name>. I'm constructing the <maxima-version>/... part of it. I wonder if it's possible to tell ASDF it doesn't need to put in the <path-to-source-file> part.
Thanks for your help, and if you have any comments, I would be interested to hear about it.
best
Robert Dodier
Dear Robert,
I've made some more progress with a Maxima extension for ASDF (attached).
This looks much more like it. Congrats!
At this point the only thing I wish I could change (and this is a minor item) is that output file paths are something like $HOME/.cache/common-lisp/<lisp-version>/<path-to-source-file>/<maxima-version>/<system-name>/<file-name>. I'm constructing the <maxima-version>/... part of it. I wonder if it's possible to tell ASDF it doesn't need to put in the <path-to-source-file> part.
You can do your own arbitrary translations in your output-files, and return (values (list ...) t)
I recommend you "just" prepend your maxima-version to the component's relative directory, or append it to its directory always, and let the usual translations take your from there. That would fit the ASDF cache model better, and thus be more clash-proof. These files are hidden from users, anyway. Also, on many implementations, lisp compilation has more than one output.
What you really need to do is: (1) inherit from cl-source-file, not source-file, so you can reuse its compile-op (2) have compile-op depend on a new operation translate-op that does just the translation step. (3) Have a generic method: (defmethod output-files ((o translate-op) (c maxima-file)) (list (merge-pathnames* (make-pathname :directory `(:relative ,(strcat "maxima-" maxima::*autoconf-version*)) :type "lisp") (component-pathname c))))
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Anarchy is order, government is civil war. — Anselme Bellegarrigue
On Wed, Sep 23, 2015 at 2:39 PM, Faré fahree@gmail.com wrote:
What you really need to do is:
Thanks, I will take a look at that. It is working well enough for the moment, but I will circle back later to polish it up.
I have a question now about how to ensure that the :maxima-file component type is defined before it is used. Let's say I have foo.asd which has :maxima-file components. Ideally I would like to simply say (asdf:load-system :foo) without having to load the maxima-file definition before. It occurred to me to put (defsystem foo :depends-on ("maxima-file") ...) and move the :maxima-file stuff into another system. But that doesn't work, of course, because :maxima-file is not yet defined before the :foo system is parsed, and I get an error about :maxima-file being unrecognized.
I can put (load-system :maxima-file) before (defsystem foo ...) and then it works OK. I'm happy that it works, but it seems a little messy to me -- there are now two kinds of dependencies, one kind that has to be loaded before the defsystem, and the other after ...
It's not a big deal, but can anyone think of a way to move the maxima-file dependency back into the :depends-on list?
Thanks for your help,
Robert Dodier
That's what :defsystem-depends-on is for.
Please let me know if you don't find it adequately documented in the manual...
Sent from my iPad
On Sep 24, 2015, at 19:41, Robert Dodier robert.dodier@gmail.com wrote:
On Wed, Sep 23, 2015 at 2:39 PM, Faré fahree@gmail.com wrote:
What you really need to do is:
Thanks, I will take a look at that. It is working well enough for the moment, but I will circle back later to polish it up.
I have a question now about how to ensure that the :maxima-file component type is defined before it is used. Let's say I have foo.asd which has :maxima-file components. Ideally I would like to simply say (asdf:load-system :foo) without having to load the maxima-file definition before. It occurred to me to put (defsystem foo :depends-on ("maxima-file") ...) and move the :maxima-file stuff into another system. But that doesn't work, of course, because :maxima-file is not yet defined before the :foo system is parsed, and I get an error about :maxima-file being unrecognized.
I can put (load-system :maxima-file) before (defsystem foo ...) and then it works OK. I'm happy that it works, but it seems a little messy to me -- there are now two kinds of dependencies, one kind that has to be loaded before the defsystem, and the other after ...
It's not a big deal, but can anyone think of a way to move the maxima-file dependency back into the :depends-on list?
Thanks for your help,
Robert Dodier
On 9/23/15 Sep 23 -3:55 PM, Robert Dodier wrote:
Hey everybody,
I've made some more progress with a Maxima extension for ASDF (attached). At this point it works pretty much as expected, for the simple examples I've tried. I think you should be able to use in Maxima like this: load("maxima_asdf.lisp"); asdf_load("foo"); to load foo.asd, which may contain :maxima-file components. asdf_load translates :maxima-files and compiles the resulting Lisp, while asdf_load_source just interprets the :maxima-files.
At this point the only thing I wish I could change (and this is a minor item) is that output file paths are something like $HOME/.cache/common-lisp/<lisp-version>/<path-to-source-file>/<maxima-version>/<system-name>/<file-name>. I'm constructing the <maxima-version>/... part of it. I wonder if it's possible to tell ASDF it doesn't need to put in the <path-to-source-file> part.
Thanks for your help, and if you have any comments, I would be interested to hear about it.
Robert, please consider writing up your experience, even as a short bulleted list of items, with information about what was *missing* from the documentation. That will help us make this easier for the next person.
thanks! r
-
Faré -
Robert Brown -
Robert Dodier -
Robert Goldman -
Robert P. Goldman