Hi,
After looking through various web pages, emails, etc, I gather it's possible to load all the fasls for some program from a jar via asdf. However, I tried the obvious things and I couldn't get it to work. Can someone explain how it's supposed to work?
I have all the fasls for Maxima in a subdirectory. (require 'asdf) (asdf:operate 'asdf:load-op :maxima) has the desired effect. So far, so good.
I put all the fasls in a jar (either in the subdirectory or without the subdirectory), and then (pushnew "jar:file:/path/to/my.jar" asdf:*central-repository*) But when I try (require 'asdf) (asdf:operate 'asdf:load-op :maxima) I eventually get a message about trying to find the first Lisp file mentioned in the maxima.asd, which I'm pretty sure it didn't find the fasls.
Can someone explain how loading all of the fasls from a jar is supposed to work? Or any other method of conglomerating fasls; for example, is it possible to combine multiple fasls into one?
Thanks for any info,
Robert Dodier
On 8/23/10 4:53 PM, Robert Dodier wrote:
Hi,
After looking through various web pages, emails, etc, I gather it's possible to load all the fasls for some program from a jar via asdf. However, I tried the obvious things and I couldn't get it to work. Can someone explain how it's supposed to work?
I have all the fasls for Maxima in a subdirectory. (require 'asdf) (asdf:operate 'asdf:load-op :maxima) has the desired effect. So far, so good.
I put all the fasls in a jar (either in the subdirectory or without the subdirectory), and then (pushnew "jar:file:/path/to/my.jar" asdf:*central-repository*) But when I try (require 'asdf) (asdf:operate 'asdf:load-op :maxima) I eventually get a message about trying to find the first Lisp file mentioned in the maxima.asd, which I'm pretty sure it didn't find the fasls.
A JAR-PATHNAME needs a reference to the directory to refer to "inside" a jar which is distinguished by the "!/" character sequence, so the form must always be
jar:URL!/PATH
where URL is usually a file URL (i.e. "file:/path/to/my.jar", although it can refer to an "http", "ftp" or any other scheme that the JVM will pass through the java.net.URL constructor). The PATH may be the empty string in which case it refers to the root directory inside the jar, but otherwise it is a "normal" namestring for a PATHNAME that refers to a sub-directory and/or a file. This nomenclature is taken from the [defacto standard as described in the java.net.JarUrlConnection javadocs][1].
[1]: http://webcache.googleusercontent.com/search?q=cache:PaMdUkE7-5MJ:download.o...
For your example, assuming you have packaged up Maxima in the directory "maxima" within the jar (i.e. there is a file "maxima/maxima.asd" in the archive, you would need to slightly modify your recipe to (pushnew "jar:file/path/to/my.jar!/maxima/" asdf:*central-repository*)
note the usual trailing '/' to denote a directory to the CL PATHNAME machinery.
To LOAD a given file within a JAR,
(load "jar:file:/path/to/my.jar!/foo")
will search the "root" directory inside the jar for the usual suspects "foo.abcl" and "foo.lisp"
[…]
Thanks, Mark. I still can't get it to work.
I tried a simpler example.
$ jar tvf foo.jar 0 Mon Aug 23 23:32:52 MDT 2010 META-INF/ 71 Mon Aug 23 23:32:52 MDT 2010 META-INF/MANIFEST.MF 0 Mon Aug 23 23:32:32 MDT 2010 systemfoo/ 1618 Mon Aug 23 23:32:02 MDT 2010 systemfoo/foobar.abcl 47 Mon Aug 23 23:26:32 MDT 2010 systemfoo/foo.asd
$ cat systemfoo/foo.asd (defsystem foo :components ((:file "foobar")))
$ cat ~/tmp/foobar.lisp (defun foobar () (print "HELLO, WORLD!"))
I created foobar.abcl via (compile-file "foobar.lisp").
So with all of that in place ...
$ java -cp maxima-abcl-fasls.jar -jar ~/tmp/abcl-0.22.0-dev.jar Armed Bear Common Lisp 0.22.0-dev Java 1.5.0_01 Sun Microsystems Inc. Java HotSpot(TM) Client VM Low-level initialization completed in 4.505 seconds. Startup completed in 13.442 seconds. Type ":help" for a list of available commands. CL-USER(1): (require 'asdf) ("CLOS" "PRINT-OBJECT" "FORMAT" "ASDF") CL-USER(2): (pushnew "jar:file:/tmp/foo.jar!/systemfoo/" asdf:*central-registry*) ("jar:file:/tmp/foo.jar!/systemfoo/") CL-USER(3): (asdf:operate 'asdf:load-op :foo) ; loading system definition from jar:file:/tmp/foo.jar!/systemfoo/foo.asd into #<PACKAGE "ASDF0"> ; registering #<SYSTEM FOO> as FOO #<THREAD "interpreter" {1CD1CEA}>: Debugger invoked on condition of type FILE-ERROR The file "jar:file:/tmp/foo.jar!/systemfoo/foobar.lisp" does not exist.
I guess ASDF didn't see foobar.abcl in the jar file. Can you tell what's going on here?
Pretty sure that's the same problem that I got with Maxima too -- not loading the .abcl's and complaining instead about the missing .lisp's.
Thanks so much for your help,
Robert Dodier
On 8/24/10 7:42 AM, Robert Dodier wrote:
[…]
$ jar tvf foo.jar 0 Mon Aug 23 23:32:52 MDT 2010 META-INF/ 71 Mon Aug 23 23:32:52 MDT 2010 META-INF/MANIFEST.MF 0 Mon Aug 23 23:32:32 MDT 2010 systemfoo/ 1618 Mon Aug 23 23:32:02 MDT 2010 systemfoo/foobar.abcl 47 Mon Aug 23 23:26:32 MDT 2010 systemfoo/foo.asd
[…]
CL-USER(1): (require 'asdf) ("CLOS" "PRINT-OBJECT" "FORMAT" "ASDF") CL-USER(2): (pushnew "jar:file:/tmp/foo.jar!/systemfoo/" asdf:*central-registry*) ("jar:file:/tmp/foo.jar!/systemfoo/") CL-USER(3): (asdf:operate 'asdf:load-op :foo) ; loading system definition from jar:file:/tmp/foo.jar!/systemfoo/foo.asd into #<PACKAGE "ASDF0"> ; registering #<SYSTEM FOO> as FOO #<THREAD "interpreter" {1CD1CEA}>: Debugger invoked on condition of type FILE-ERROR The file "jar:file:/tmp/foo.jar!/systemfoo/foobar.lisp" does not exist.
I guess ASDF didn't see foobar.abcl in the jar file. Can you tell what's going on here?
Actually, if you look at the error, ASDF is looking for 'foobar.lisp' to see if it is out of date with respect to the 'foobar.abcl' FASL. With ASDF2, you just need to package the source in the jar as there is a new mechanism for establishing links between the source and the compiled in the ASDF-OUTPUT-TRANSLATIONS machinery, which by default writes all FASLs under "~/.cache/common-lisp".
If your intention is to provide a jar archive containing the entire Maxima system ready to run without further compilation, you'll need to fiddle with the source-to-fasl translation mechanism as documented in the [ASDF manual][1], but as a shortcut convince yourself that things will work from a jar, you can probably start with calling ASDF:DISABLE-OUTPUT-TRANSLATIONS before you invoke the ASDF:LOAD-OP.
[1]: http://common-lisp.net/project/asdf/asdf/Controlling-where-ASDF-saves-compil...
I remember some traffic on the ASDF mailing list a couple months ago about creating pure binary ADSF definitions in the context of distributing systems for ECL, for which a runtime without gcc means no compiler.
And maybe Alan Ruttenberg has a more advanced solution here as he had (from what I remember) experimental code to dump all current ASDF definitions in a running ABCL image to a jar file that could be restarted. I believe if you search the armedbear-devel archive there might be a reference to this work.
--- On Tue, 8/24/10, Mark Evenson evenson@panix.com wrote:
Actually, if you look at the error, ASDF is looking for 'foobar.lisp' to see if it is out of date with respect to the 'foobar.abcl' FASL. With ASDF2, you just need to package the source in the jar as there is a new mechanism for establishing links between the source and the compiled in the ASDF-OUTPUT-TRANSLATIONS machinery, which by default writes all FASLs under "~/.cache/common-lisp".
I dunno. ASDF's behavior seems kind of crazy. Whether the .abcl is present has no effect; if the source is present, ASDF compiles it and writes it to the cache, otherwise it barfs out an error. Doesn't that seem strange? Wouldn't one expect that if the .abcl is present, and demonstrably up to date, that the .abcl gets loaded and the .lisp is ignored?
If your intention is to provide a jar archive containing the entire Maxima system ready to run without further compilation, you'll need to fiddle with the source-to-fasl translation mechanism as documented in the [ASDF manual][1], but as a shortcut convince yourself that things will work from a jar, you can probably start with calling ASDF:DISABLE-OUTPUT-TRANSLATIONS before you invoke the ASDF:LOAD-OP.
I called (asdf:disable-output-translations) but it had no effect.
I remember some traffic on the ASDF mailing list a couple months ago about creating pure binary ADSF definitions in the context of distributing systems for ECL, for which a runtime without gcc means no compiler.
Well, that sounds interesting. I guess I can hope that they'll release something eventually.
And maybe Alan Ruttenberg has a more advanced solution here as he had (from what I remember) experimental code to dump all current ASDF definitions in a running ABCL image to a jar file that could be restarted. I believe if you search the armedbear-devel archive there might be a reference to this work.
Searched the list archives back to Jan 2010, no luck. Found a lot of stuff about ASDF and jars but I couldn't identify any of it as something about restartable jars. Thanks for the tip anyway.
best
Robert Dodier
armedbear-devel@common-lisp.net
-
David Zhang -
Mark Evenson -
Robert Dodier