#365: ABCL-Jar not working on Mac OS X ----------------------------+----------------- Reporter: robert goldman | Owner: Type: defect | Status: new Priority: major | Milestone: Component: abcl-contrib | Version: Keywords: | ----------------------------+----------------- When I tried to package a system of mine, I got cryptic errors and an empty jar file.
Turning on debugging, I see this: {{{ PRISM> (asdf-jar:package :abcl-prism) Packaging ASDF definition of #<ASDF/SYSTEM:SYSTEM "abcl-prism"> Packaging contents in /var/tmp/abcl-prism-all.jar with recursive dependencies ITERATE, JSS. /Users/rpg/prismatic/code/quiksilver/abcl/package.lisp => abcl-prism/package.lisp /Users/rpg/.cache/common-lisp/abcl-1.4.0-dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/package.abcl => Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go ; Evaluation aborted on #<SIMPLE-ERROR {58717F69}>. PRISM> }}} I can't say for sure, but it looks suspiciously like the results of ASDF output translations might be causing ASDF-JAR to fail. Is there any chance that these pathnames are simply too long? Or contain characters that ABCL treats as illegal?
What's the appropriate solution for packaging up fasls so that I can load them from jars (I have no idea how else to package up Lisp so that it can be called from Java...)?
BTW, there is a long-standing, unresolved bug in the ABCL implementation of the ASDF bundling operations on Mac OSX, so that's not a solution. Thanks!
-- Ticket URL: http://abcl.org/trac/ticket/365 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------- Reporter: robert goldman | Owner: Type: defect | Status: new Priority: major | Milestone: Component: abcl-contrib | Version: Resolution: | Keywords: -----------------------------+-----------------
Comment (by robert goldman):
Tracing `MAKE-PATHNAME` shows an error, but I'm not sure what it is: {{{ /Users/rpg/prismatic/code/quiksilver/abcl/package.lisp => abcl-prism/package.lisp 0: (MAKE-PATHNAME :DIRECTORY (:ABSOLUTE) :NAME NIL :TYPE NIL :VERSION NIL :DEFAULTS #P"/Users/rpg/prismatic/code/quiksilver/abcl/package.lisp") 0: MAKE-PATHNAME returned #P"/" 0: (MAKE-PATHNAME :HOST NIL :DEVICE :UNSPECIFIC :DIRECTORY (:ABSOLUTE :WILD-INFERIORS) :NAME :WILD :TYPE :WILD :VERSION NIL) 0: MAKE-PATHNAME returned #P"/**/*.*" 0: (MAKE-PATHNAME :HOST NIL :DEVICE NIL :DIRECTORY (:ABSOLUTE "Users" "rpg" ".cache" "common-lisp" "abcl-1.4.0-dev-fasl42-macosx-x64" "Users" "rpg" "prismatic" "code" "quiksilver" "abcl") :NAME "package" :TYPE "lisp" :VERSION NIL) 0: MAKE-PATHNAME returned #P"/Users/rpg/.cache/common-lisp/abcl-1.4.0 -dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/package.lisp" 0: (MAKE-PATHNAME :DEFAULTS #P"/Users/rpg/.cache/common-lisp/abcl-1.4.0 -dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/package.lisp" :TYPE "abcl") 0: MAKE-PATHNAME returned #P"/Users/rpg/.cache/common-lisp/abcl-1.4.0 -dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/package.abcl" 0: (MAKE-PATHNAME :DEFAULTS #P"abcl-prism/package.lisp" :TYPE "abcl" :DIRECTORY ("abcl-prism")) 0: MAKE-PATHNAME returned Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go Serious printing error: org.armedbear.lisp.Go }}}
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:1 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------- Reporter: robert goldman | Owner: Type: defect | Status: new Priority: major | Milestone: Component: abcl-contrib | Version: Resolution: | Keywords: -----------------------------+-----------------
Comment (by robert goldman):
The following causes `make-pathname` to error out: {{{ (make-pathname :defaults (make-pathname :directory "abcl-prism" :name "package" :type "lisp") :type "abcl" :directory '("abcl-prism")) }}}
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:2 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------- Reporter: robert goldman | Owner: Type: defect | Status: new Priority: major | Milestone: Component: abcl-contrib | Version: Resolution: | Keywords: -----------------------------+-----------------
Comment (by robert goldman):
I have further narrowed the problem, I believe. I find the following in the code: {{{ (output-entry (make-pathname :defaults source-entry :type "abcl" :directory (append root (rest (pathname-directory source- entry)))))) }}} In my usage, `root` is `NIL`, and the call to `REST` rips the `:RELATIVE` qualifier off the pathname-directory of `SOURCE-ENTRY`. This leaves me with a `("abcl-prism")` as the pathname directory component, which ABCL is rejecting as ill-formed.
I guess I don't see why this would ever work, if `ROOT` is left with the default `NIL` value.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:3 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------- Reporter: robert goldman | Owner: Type: defect | Status: new Priority: major | Milestone: Component: abcl-contrib | Version: Resolution: | Keywords: -----------------------------+-----------------
Comment (by mevenson):
I'm seeing problems as well, so this seems to a more general problem
What's your LISP-IMPLEMENTATION-VERSION? Mine is: {{{ CL-USER> (lisp-implementation-version)
"1.4.0-dev" "Java_HotSpot(TM)_64-Bit_Server_VM-Oracle_Corporation-1.8.0_11-b12" "x86_64-Mac_OS_X-10.9.4" }}}
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:4 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+---------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: accepted Priority: major | Milestone: Component: abcl-contrib | Version: Resolution: | Keywords: -----------------------------+---------------------- Changes (by mevenson):
* owner: => mevenson * status: new => accepted
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:5 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+---------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: Component: abcl-contrib | Version: Resolution: fixed | Keywords: -----------------------------+---------------------- Changes (by mevenson):
* status: accepted => closed * resolution: => fixed
Comment:
Eduardo Bellino seems to have addressed this in r14717. Please try a new 1.4.0-dev to see if it solves your problem.
http://abcl.org/trac/changeset/14717
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:6 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: fixed | Keywords: -----------------------------+----------------------- Changes (by mevenson):
* version: => 1.4.0-dev * milestone: => 1.4.0
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:7 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: reopened Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: | Keywords: -----------------------------+----------------------- Changes (by robert goldman):
* status: closed => reopened * resolution: fixed =>
Comment:
I have just updated from SVN and rebuilt, and the asdf-jar packaging still fails for me, although it fails with a different error now. Here's what I see in the slime repl: {{{ luation aborted on #<FILE-ERROR {2B53822}>. CL-USER> (asdf-jar:package "abcl-prism" :recursive nil)
Packaging ASDF definition of #<ASDF/SYSTEM:SYSTEM "abcl-prism"> Packaging contents in /var/tmp/abcl-prism.jar }}} And here's the debugger window contents: {{{ Unsupported directory component "abcl-prism". [Condition of type FILE-ERROR]
Restarts: 0: [RETRY] Retry SLIME REPL evaluation request. 1: [*ABORT] Return to SLIME's top level. 2: [ABORT] Abort thread.
Backtrace: 0: (#<FUNCTION {2D5FD903}> #<FILE-ERROR {331A2257}> #<FUNCTION {2D5FD903}>) 1: (APPLY #<FUNCTION {2D5FD903}> (#<FILE-ERROR {331A2257}> #<FUNCTION {2D5FD903}>)) 2: (SYSTEM::RUN-HOOK SYSTEM::*INVOKE-DEBUGGER-HOOK* #<FILE-ERROR {331A2257}> #<FUNCTION {2D5FD903}>) 3: (INVOKE-DEBUGGER #<FILE-ERROR {331A2257}>) 4: org.armedbear.lisp.Lisp.error(Lisp.java:382) 5: org.armedbear.lisp.Pathname.getDirectoryNamestring(Pathname.java:808) 6: org.armedbear.lisp.Pathname.getNamestring(Pathname.java:700) 7: org.armedbear.lisp.zip.execute(zip.java:220) 8: org.armedbear.lisp.zip.execute(zip.java:68) 9: org.armedbear.lisp.Symbol.execute(Symbol.java:814) 10: org.armedbear.lisp.LispThread.execute(LispThread.java:832) 11: org.armedbear.lisp.asdf_jar_TMP_1.execute(asdf-jar.lisp:15) 12: org.armedbear.lisp.CompiledClosure.execute(CompiledClosure.java:121) 13: org.armedbear.lisp.LispThread.execute(LispThread.java:851) 14: org.armedbear.lisp.Lisp.evalCall(Lisp.java:589) 15: org.armedbear.lisp.Lisp.eval(Lisp.java:540) 16: org.armedbear.lisp.Primitives$pf__eval.execute(Primitives.java:345) 17: (SYSTEM:ZIP #P"/var/tmp/abcl-prism.jar" #<EQUAL HASH-TABLE 7 entries, 11 buckets {78C497F6}>) 18: (PACKAGE "abcl-prism" :RECURSIVE NIL) 19: (SYSTEM::%EVAL (PACKAGE "abcl-prism" :RECURSIVE NIL)) --more-- }}} The hashtable contents are as follows: {{{ #<HASH-TABLE {78C497F6}> -------------------- Count: 7 Size: 11 Test: EQUAL Rehash size: 1.5 Rehash threshold: 0.75 [clear hashtable] Contents: #P"/Users/rpg/prismatic/code/quiksilver/abcl/abcl-prism.asd" = #P"abcl- prism/abcl-prism.asd" [remove entry] #P"/Users/rpg/.cache/common-lisp/abcl-1.4.0-dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/package.abcl" = #<PATHNAME <<error printing object>> {33A72123}> [remove entry] #P"/Users/rpg/.cache/common-lisp/abcl-1.4.0-dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/utils.abcl" = #<PATHNAME <<error printing object>> {21709B53}> [remove entry] #P"/Users/rpg/prismatic/code/quiksilver/abcl/package.lisp" = #P"abcl- prism/package.lisp" [remove entry] #P"/Users/rpg/prismatic/code/quiksilver/abcl/utils.lisp" = #P"abcl- prism/utils.lisp" [remove entry] #P"/Users/rpg/prismatic/code/quiksilver/abcl/add-script.lisp" = #P"abcl- prism/add-script.lisp" [remove entry] #P"/Users/rpg/.cache/common-lisp/abcl-1.4.0-dev- fasl42-macosx-x64/Users/rpg/prismatic/code/quiksilver/abcl/add- script.abcl" = #<PATHNAME <<error printing object>> {67E5774B}> [remove entry] }}}
I note that the mapped pathnames for all of the compiled files gives an error on printing, although the source files are all ok.
Unfortunately, when I attempt to do anything with the bad pathnames, even `describe`, it causes a crash in some print method. However, I have determined that the `pathname-directory` returned from one of these is `"abcl-prism"`, and the `pathname-directory` of a working pathname is `(:relative "abcl-prism")`, so I believe there's still a mapping bug.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:8 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: reopened Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: | Keywords: -----------------------------+-----------------------
Comment (by robert goldman):
When I look at this further, I see that the logic for mapping the compiled files has not materially changed. The `root` entry is still appended, even if it is null, to the second entry in the pathname-directory of the source-entry. I am not sure why the second element alone is correct, but it's clear that it won't be either `:relative` or `:absolute`, so the mapped directory will still be incorrect if the root is `NIL`.
Question: if the root is `NIL`, should we simply take the first element of the source-mapping in its place?
It would really help to add a comment or two to the logic of the output- entry binding.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:9 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: reopened Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: | Keywords: -----------------------------+-----------------------
Comment (by robert goldman):
Is there some reason that the source-entry is translated into the output- entry by taking all of it as defaults, rather than just the name? I'm wondering if the directory components are combining in some anomalous way when computing the `output-entry`.
Again, if there was an explanation of what it was intended to compute, this would be easier to debug.
Thanks!
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:10 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: reopened Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: | Keywords: -----------------------------+-----------------------
Comment (by robert goldman):
Put more clearly, are we trying to put the .abcl files next to the source files, or are we trying to put them in some structure that mirrors the structure of the ASDF cache according to Fare's output translations logic?
If it's the former, I'm stumped by the `CADR` invocation.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:11 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: reopened Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: | Keywords: -----------------------------+-----------------------
Comment (by mevenson):
We are mapping from the location of source pathnames to destination pathnames in the jar artifact.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:12 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: fixed | Keywords: -----------------------------+----------------------- Changes (by robert goldman):
* status: reopened => closed * resolution: => fixed
Comment:
Sorry for the noise. This was entirely my bad. I ended up with a stale abcl-contrib.jar lying around in my source path, which blocked seeing the good code.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:13 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: fixed | Keywords: -----------------------------+-----------------------
Comment (by robert goldman):
One last question if you don't mind: there seems like a Catch-22 about using these jar files. You have to add them to the ASDF path using the asdf-jar contrib, but how do you know where they are? Seems like the asdf- contrib.jar has a cleaner solution where you can simply `(require :asdf- contrib.jar)`, and all the systems in the jar file can become loadable. Is there something like this one can do with a jar file make with ASDF-JAR?
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:14 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: fixed | Keywords: -----------------------------+-----------------------
Comment (by mevenson):
Replying to [comment:14 robert goldman]:
One last question if you don't mind: there seems like a Catch-22 about
using these jar files. You have to add them to the ASDF path using the asdf-jar contrib, but how do you know where they are? Seems like the asdf- contrib.jar has a cleaner solution where you can simply `(require :asdf- contrib)`, and all the systems in the jar file can become loadable. Is there something like this one can do with a jar file make with ASDF-JAR?
The jar is 'abcl-contrib.jar' located via {{{(require :abcl-contrib)}}}. This is considered a "system jar" to be distributed with 'abcl.jar', which would usually be in a location not directly writable by the user.
One possibility would be if you are looking to distribute a standalone application, simply adding such packages to {{{abcl-contrib.jar}}}.
But for locating ASDF systems in general, I suppose that the right way forward would be to use the ASDF configuration DSL to provide the location of your systems, which just happen to be packaged in jar files. ABCL uses a EXT:JAR-PATHNAME to represent such locations which is an extension of CL:PATHNAME, so if we were to ensure that the ASDF configuration DSL can deal with such extensions, things might be able to work. Now if only I could find an ASDF maintainer to talk to about this…
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:15 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X -----------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: 1.4.0 Component: abcl-contrib | Version: 1.4.0-dev Resolution: fixed | Keywords: -----------------------------+-----------------------
Comment (by robert goldman):
Let me start a new ticket for this, since I have dragged us way off topic.
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:16 armedbear http://abcl.org armedbear
#365: ABCL-Jar not working on Mac OS X ---------------------------------+----------------------- Reporter: robert goldman | Owner: mevenson Type: defect | Status: closed Priority: major | Milestone: 1.3.2 Component: abcl-contrib | Version: 1.4.0-dev Resolution: fixed | Keywords: Parent Tickets: | ---------------------------------+----------------------- Changes (by mevenson):
* milestone: 1.4.0 => 1.3.2
-- Ticket URL: http://abcl.org/trac/ticket/365#comment:17 armedbear http://abcl.org armedbear
armedbear-ticket@common-lisp.net
-
armedbear