On 6/9/11 16:37 , Faré wrote: […]
Can you write a simple test to be added to test/asdf-pathnames.script,
I'll work on that. Can I just create a new top-level form or do you prefer I wedge it into TEST-COMPONENT-PATHNAMES with an #+abcl/#-abcl conditional?
and/or send me an example such that I can reproduce the bug at home? Maybe just a trace output of the relevant functions: (trace translate-pathname* and compile-file-pathname* apply-output-translations) (add any function there necessary to show where it breaks).
A trace for the JSS system packaged in abcl-contrib.jar:
CL-USER> (asdf:compile-system :jss :force t) 0: (ASDF:APPLY-OUTPUT-TRANSLATIONS #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl") 1: (ASDF::TRANSLATE-PATHNAME* #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl" #P"jar:file:/**/*.jar!/**/*.*" #<FUNCTION ASDF::TRANSLATE-JAR-PATHNAME {66EF7D74}> NIL #P"jar:file:/**/*.jar!/**/*.*") 2: (ASDF:APPLY-OUTPUT-TRANSLATIONS #P"/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl") 3: (ASDF::TRANSLATE-PATHNAME* #P"/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" #P"/___jar___file___root___/**/*.*" #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*" NIL #P"/___jar___file___root___/**/*.*") 3: TRANSLATE-PATHNAME* returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 2: APPLY-OUTPUT-TRANSLATIONS returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 1: TRANSLATE-PATHNAME* returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 0: APPLY-OUTPUT-TRANSLATIONS returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 0: (ASDF:APPLY-OUTPUT-TRANSLATIONS #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl") 1: (ASDF::TRANSLATE-PATHNAME* #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl" #P"jar:file:/**/*.jar!/**/*.*" #<FUNCTION ASDF::TRANSLATE-JAR-PATHNAME {66EF7D74}> NIL #P"jar:file:/**/*.jar!/**/*.*") 2: (ASDF:APPLY-OUTPUT-TRANSLATIONS #P"/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl") 3: (ASDF::TRANSLATE-PATHNAME* #P"/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" #P"/___jar___file___root___/**/*.*" #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*" NIL #P"/___jar___file___root___/**/*.*") 3: TRANSLATE-PATHNAME* returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 2: APPLY-OUTPUT-TRANSLATIONS returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 1: TRANSLATE-PATHNAME* returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 0: APPLY-OUTPUT-TRANSLATIONS returned #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" 0: (ASDF:COMPILE-FILE-PATHNAME* #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.lisp" :OUTPUT-FILE #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" :OUTPUT-FILE #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl") 0: COMPILE-FILE-PATHNAME* returned #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl"
And the subsequent COMPILE-FILE fails as it can't write to the jar file returned by COMPILE-FILE-PATHNAME*.
Note that when I (describe #p"jar:file:///foo/bar.jar!/baz/quux.lisp") I get: #P"jar:file:/foo/bar.jar!/baz/quux.lisp" is an object of type EXTENSIONS:JAR-PATHNAME: HOST NIL DEVICE (#P"/foo/bar.jar") DIRECTORY (:ABSOLUTE "baz") NAME "quux" TYPE "lisp" VERSION NIL
I believe that a pathname instead of a namestring makes that non-conformant. Sigh.
You mean the list with pathname in the device field? Yes, ABCL is non-conformant in this manner in order to support addressing entries in jar files.
Note that if requires you could keep a cache of the pathname in addition (or underlying store?) to any namestring in the device.
I don't think I understand this suggestion.
[…]
Recall that ABCL uses the presence of a cons in the DEVICE component to indicate that a Pathname represents an entry in a jar archive. A pathname located within a jar archive is not a writable location, so we introduced ABCL specific ASDF output translation rules which translate such a source location to an appropriately differentiated output location in the ASDF cache hierarchy. This results in ASDF making the following kind of call in preparation of the output file location it will pass to COMPILE-FILE:
(compile-file-pathname* <path with a DEVICE> :output-file<path without a DEVICE>)
What output-file do you get and where is it from?
0: (ASDF:COMPILE-FILE-PATHNAME* #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.lisp" :OUTPUT-FILE #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl" :OUTPUT-FILE #P"/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl") 0: COMPILE-FILE-PATHNAME* returned #P"jar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl"
The arg to :output-file has come from the OUTPUT-FILES :around method passing in "jar:file:…" and then hitting the TRANSLATE-JAR-PATHNAME rule.
Is any path there not absolute, and if so why?
No, it is always absolute as TRANSLATE-JAR-PATHNAME returns the fully translated pathname.
If there were some mechanism to indicate that the invocation of APPLY-OUTPUT-TRANSLATIONS had really "finished" so that no further processing was necessary, we could possibly plausibly short-circuit this call to COMPILE-FILE-PATHAME*.
Well, output-files accepts a secondary value of T to specify that no more translations are needed.
I missed that: I'll look into this further when I get a chance.
I don't understand yet what to change into ASDF, but either we can somehow reuse that mechanism, or we can provide something similar inside apply-output-translations, or somehow be more clever in translate-jar-pathname.
I suppose my recursive invocation of APPLY-OUTPUT-TRANSLATIONS in the ABCL-specific TRANSLATE-JAR-PATHNAME adds the missing TYPE to the pathname which other code paths don't have at this point, which is why the call to COMPILE-FILE-PATHNAME* is necessary.
My head hurts. What type is added to what where?
My understanding is that ASDF uses the COMPILE-FILE-PATHNAME* to supply the TYPE of fasl, which is implementation dependent. Otherwise I didn't have an explanation as to why we run COMPILE-FILE-PATHNAME* before COMPILE-FILE which is going to return the same results, as in my testing the value that :output-file has at this point is already the final value anyways.
Can you give examples of functions giving the "wrong" results?
No. As I stated, it is correct that COMPILE-FILE-PATHNAME* merges in the device from input-file when the output-file arg has NIL for device. This is the expected behavior from the ANSI perspective, but what we are trying to get away from in ABCL's case.
Thanks for thinking with me on this.