[armedbear-devel] r12666 breaks ASDF compilation on Linux
I just tried abcl 0.20 with Snow, and compilation (done with asdf:compile-op) fails early with: Compilation failed: Not an absolute pathname #P"\\___jar___file___root___\\**\\*.*" r11664 works fine. Sorry that I didn't check before the release, I remembered checking it not much time ago but evidently I was wrong (r12666 is two weeks old). Alessio
On 5/28/10 9:36 PM, Alessio Stalla wrote:
I just tried abcl 0.20 with Snow, and compilation (done with asdf:compile-op) fails early with:
Compilation failed: Not an absolute pathname #P"\\___jar___file___root___\\**\\*.*"
r11664 works fine. Sorry that I didn't check before the release, I remembered checking it not much time ago but evidently I was wrong (r12666 is two weeks old).
I broke my wrist last Thursday, just getting back from the hospital, so getting to the bottom of this is going to be slow on my part. You say that this is "on Linux" that this fails, yet your Pathname looks like a Windows one. Do you really get the error you pasted from trying to ASDF compile Snow on Linux? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Mon, May 31, 2010 at 7:52 AM, Mark Evenson <evenson@panix.com> wrote:
On 5/28/10 9:36 PM, Alessio Stalla wrote:
I just tried abcl 0.20 with Snow, and compilation (done with asdf:compile-op) fails early with:
Compilation failed: Not an absolute pathname #P"\\___jar___file___root___\\**\\*.*"
r11664 works fine. Sorry that I didn't check before the release, I remembered checking it not much time ago but evidently I was wrong (r12666 is two weeks old).
I broke my wrist last Thursday, just getting back from the hospital, so getting to the bottom of this is going to be slow on my part.
Oh, I'm sorry for you! I hope you recover quickly.
You say that this is "on Linux" that this fails, yet your Pathname looks like a Windows one. Do you really get the error you pasted from trying to ASDF compile Snow on Linux?
Yes, absolutely. I do all my development on Linux. I didn't study the problem at all, I just verified that with the previous revision of asdf.lisp it worked. Since r12666 is a single, small diff, I can try fixing it myself - probably it's just a wrong conditional clause. Take care, Alessio
Is there any workaround for this? On 31 May 2010 09:08, Alessio Stalla <alessiostalla@gmail.com> wrote:
On Mon, May 31, 2010 at 7:52 AM, Mark Evenson <evenson@panix.com> wrote:
On 5/28/10 9:36 PM, Alessio Stalla wrote:
I just tried abcl 0.20 with Snow, and compilation (done with asdf:compile-op) fails early with:
Compilation failed: Not an absolute pathname #P"\\___jar___file___root___\\**\\*.*"
r11664 works fine. Sorry that I didn't check before the release, I remembered checking it not much time ago but evidently I was wrong (r12666 is two weeks old).
I broke my wrist last Thursday, just getting back from the hospital, so getting to the bottom of this is going to be slow on my part.
Oh, I'm sorry for you! I hope you recover quickly.
You say that this is "on Linux" that this fails, yet your Pathname looks like a Windows one. Do you really get the error you pasted from trying to ASDF compile Snow on Linux?
Yes, absolutely. I do all my development on Linux.
I didn't study the problem at all, I just verified that with the previous revision of asdf.lisp it worked. Since r12666 is a single, small diff, I can try fixing it myself - probably it's just a wrong conditional clause.
Take care, Alessio
_______________________________________________ armedbear-devel mailing list armedbear-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
-- Knut Olav Bøhmer
On 6/1/10 9:50 PM, Knut Olav Bøhmer wrote:
Is there any workaround for this?
You got a patch for my wrist? ;} But seriously, I think disabling the JAR output translation by removing the reference(s) in ASDF:*OUTPUT-TRASLATIONS* with something like (setf asdf:*output-translations (list (remove #P"/___jar___file___root___/**/*.*" (first *output-translations*) :key #'car :test #'equal)) would work. Alessio: snow-0.3 fails to compile with 'sys:variable-information' not being found in 'snow.lisp', so I can't reproduce your error. Maybe you need a new release from Snow trunk? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
2010/6/2 Mark Evenson <evenson@panix.com>:
On 6/1/10 9:50 PM, Knut Olav Bøhmer wrote:
Is there any workaround for this?
You got a patch for my wrist? ;}
But seriously, I think disabling the JAR output translation by removing the reference(s) in ASDF:*OUTPUT-TRASLATIONS* with something like
(setf asdf:*output-translations (list (remove #P"/___jar___file___root___/**/*.*" (first *output-translations*) :key #'car :test #'equal))
would work.
Alessio: snow-0.3 fails to compile with 'sys:variable-information' not being found in 'snow.lisp', so I can't reproduce your error. Maybe you need a new release from Snow trunk?
You're right: Snow 0.3 was still supposed to build with a patched abcl, while trunk is not. However, I found out that the asdf problem is only in the binary release of ABCL downloaded from the site, while if I compile 0.20 myself it works, so r12666 is not the culprit. Since you cited a Windows pathname, and I think Erik built those binaries on Windows, is it possible that ASDF builds differently depending on the OS, so that the resulting binary only works on that OS? Cheers, Alessio
2010/6/2 Alessio Stalla <alessiostalla@gmail.com>:
2010/6/2 Mark Evenson <evenson@panix.com>:
On 6/1/10 9:50 PM, Knut Olav Bøhmer wrote:
Is there any workaround for this?
You got a patch for my wrist? ;}
But seriously, I think disabling the JAR output translation by removing the reference(s) in ASDF:*OUTPUT-TRASLATIONS* with something like
(setf asdf:*output-translations (list (remove #P"/___jar___file___root___/**/*.*" (first *output-translations*) :key #'car :test #'equal))
would work.
Alessio: snow-0.3 fails to compile with 'sys:variable-information' not being found in 'snow.lisp', so I can't reproduce your error. Maybe you need a new release from Snow trunk?
You're right: Snow 0.3 was still supposed to build with a patched abcl, while trunk is not. However, I found out that the asdf problem is only in the binary release of ABCL downloaded from the site, while if I compile 0.20 myself it works, so r12666 is not the culprit. Since you cited a Windows pathname, and I think Erik built those binaries on Windows, is it possible that ASDF builds differently depending on the OS, so that the resulting binary only works on that OS?
Answering to myself, it appears indeed that the 0.20 binary contains a reference to Windows pathnames: CL-USER(1): *features* (:JAVA-1.6 :ARMEDBEAR :ABCL :COMMON-LISP :ANSI-CL :UNIX :LINUX :CDR6) CL-USER(2): (require :asdf) ("CLOS" "PRINT-OBJECT" "FORMAT" "ASDF") CL-USER(3): (asdf::wrapping-output-translations) (:OUTPUT-TRANSLATIONS :INHERIT-CONFIGURATION (#P"\\___jar___file___root___\\**\\*.*" (:USER-CACHE #P"**\\*.*")) (#P"jar:file:\\**\\*.jar!/**/*.*" (:FUNCTION ASDF::TRANSLATE-JAR-PATHNAME)) :ENABLE-USER-CACHE) This is presumably because pathnames are serialized to FASLs using their namestring, which is not necessarily the same string that was passed in with #P"...". Storing the user-provided string in the pathname and using that as a namestring if the pathname has not been destructively modified could solve this issue, if it's doable. Sigh, I'll have to reboot to Windows... ;) Cheers, Alessio
2010/6/2 Alessio Stalla <alessiostalla@gmail.com>:
2010/6/2 Alessio Stalla <alessiostalla@gmail.com>:
2010/6/2 Mark Evenson <evenson@panix.com>:
On 6/1/10 9:50 PM, Knut Olav Bøhmer wrote:
Is there any workaround for this?
You got a patch for my wrist? ;}
But seriously, I think disabling the JAR output translation by removing the reference(s) in ASDF:*OUTPUT-TRASLATIONS* with something like
(setf asdf:*output-translations (list (remove #P"/___jar___file___root___/**/*.*" (first *output-translations*) :key #'car :test #'equal))
would work.
Alessio: snow-0.3 fails to compile with 'sys:variable-information' not being found in 'snow.lisp', so I can't reproduce your error. Maybe you need a new release from Snow trunk?
You're right: Snow 0.3 was still supposed to build with a patched abcl, while trunk is not. However, I found out that the asdf problem is only in the binary release of ABCL downloaded from the site, while if I compile 0.20 myself it works, so r12666 is not the culprit. Since you cited a Windows pathname, and I think Erik built those binaries on Windows, is it possible that ASDF builds differently depending on the OS, so that the resulting binary only works on that OS?
Answering to myself, it appears indeed that the 0.20 binary contains a reference to Windows pathnames:
CL-USER(1): *features* (:JAVA-1.6 :ARMEDBEAR :ABCL :COMMON-LISP :ANSI-CL :UNIX :LINUX :CDR6)
CL-USER(2): (require :asdf) ("CLOS" "PRINT-OBJECT" "FORMAT" "ASDF")
CL-USER(3): (asdf::wrapping-output-translations) (:OUTPUT-TRANSLATIONS :INHERIT-CONFIGURATION (#P"\\___jar___file___root___\\**\\*.*" (:USER-CACHE #P"**\\*.*")) (#P"jar:file:\\**\\*.jar!/**/*.*" (:FUNCTION ASDF::TRANSLATE-JAR-PATHNAME)) :ENABLE-USER-CACHE)
This is presumably because pathnames are serialized to FASLs using their namestring, which is not necessarily the same string that was passed in with #P"...". Storing the user-provided string in the pathname and using that as a namestring if the pathname has not been destructively modified could solve this issue, if it's doable. Sigh, I'll have to reboot to Windows... ;)
I was able to fix the problem by moving the setting of the namestring during the initialization of a pathname close to the beginning of the init method, before / characters are replaced by \ on Windows. I verified that on Windows I can load files regularly and that the binary built on Windows works on Linux as well, but since pathnames are foreign territory for me, I haven't committed the fix yet in case someone points out some obvious mistake. Attached is a patch with my fix. Bye, Alessio
On 6/2/10 4:27 PM, Alessio Stalla wrote: Index: src/org/armedbear/lisp/Pathname.java =================================================================== --- src/org/armedbear/lisp/Pathname.java (revisione 12737) +++ src/org/armedbear/lisp/Pathname.java (copia locale) @@ -232,6 +232,7 @@ if (s == null) { return; } + namestring = s; if (s.equals(".") || s.equals("./") || (Utilities.isPlatformWindows && s.equals(".\\"))) { directory = new Cons(Keyword.RELATIVE); @@ -429,7 +430,6 @@ s = System.getProperty("user.home").concat(s.substring(1)); } } - namestring = s; if (Utilities.isPlatformWindows) { if (s.length() >= 2 && s.charAt(1) == ':') { device = new SimpleString(s.charAt(0)); I think the second part of the patch is good, as Pathname.init() shouldn't be setting the namestring at all. The getNamestring() accessor should construct the value on its first invocation. So I would be led to think that the first part of the patch is wrong. At best, it has no effect on the action on the algorithim. It might be better to transform all '\' to '/' after the check for UNC paths (lines 244-267). Not sure how Windows drive letters as devices are being handled here. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Wed, Jun 2, 2010 at 6:33 PM, Mark Evenson <evenson@panix.com> wrote:
On 6/2/10 4:27 PM, Alessio Stalla wrote:
Index: src/org/armedbear/lisp/Pathname.java =================================================================== --- src/org/armedbear/lisp/Pathname.java (revisione 12737) +++ src/org/armedbear/lisp/Pathname.java (copia locale) @@ -232,6 +232,7 @@ if (s == null) { return; } + namestring = s; if (s.equals(".") || s.equals("./") || (Utilities.isPlatformWindows && s.equals(".\\"))) { directory = new Cons(Keyword.RELATIVE); @@ -429,7 +430,6 @@ s = System.getProperty("user.home").concat(s.substring(1)); } } - namestring = s; if (Utilities.isPlatformWindows) { if (s.length() >= 2 && s.charAt(1) == ':') { device = new SimpleString(s.charAt(0));
I think the second part of the patch is good, as Pathname.init() shouldn't be setting the namestring at all. The getNamestring() accessor should construct the value on its first invocation.
But won't that construct it with \ on Windows?
So I would be led to think that the first part of the patch is wrong. At best, it has no effect on the action on the algorithim.
It has the effect of preventing getNamestring() to recalculate the namestring until the pathname is modified. Alessio
On 6/2/10 6:56 PM, Alessio Stalla wrote:
On Wed, Jun 2, 2010 at 6:33 PM, Mark Evenson<evenson@panix.com> wrote: […]
I think the second part of the patch is good, as Pathname.init() shouldn't be setting the namestring at all. The getNamestring() accessor should construct the value on its first invocation.
But won't that construct it with \ on Windows?
Yeah, but that should be the right behavior: translate to the native directory separator, right?
So I would be led to think that the first part of the patch is wrong. At best, it has no effect on the action on the algorithim.
It has the effect of preventing getNamestring() to recalculate the namestring until the pathname is modified.
Hmmm: I don't see why that would be desirable. I evidently don't understand enough context here then without running tests (which is tough to do with this wrist). Naively, I expect we need some sort of heuristic in Pathname.init(String) that detects if the Pathname was serialized on the other platform, then takes corrective action. Cases like 'c:/this/path' or '\\\\server\mount\point' serialized under Windows will have no exact meaning under non-Windows for which I don't know what would be reasonable behavior. Just ignore those parts? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Wed, Jun 2, 2010 at 7:14 PM, Mark Evenson <evenson@panix.com> wrote:
On 6/2/10 6:56 PM, Alessio Stalla wrote:
On Wed, Jun 2, 2010 at 6:33 PM, Mark Evenson<evenson@panix.com> wrote: […]
I think the second part of the patch is good, as Pathname.init() shouldn't be setting the namestring at all. The getNamestring() accessor should construct the value on its first invocation.
But won't that construct it with \ on Windows?
Yeah, but that should be the right behavior: translate to the native directory separator, right?
Yes, but not when the user passed / as a directory separator.
So I would be led to think that the first part of the patch is wrong. At best, it has no effect on the action on the algorithim.
It has the effect of preventing getNamestring() to recalculate the namestring until the pathname is modified.
Hmmm: I don't see why that would be desirable. I evidently don't understand enough context here then without running tests (which is tough to do with this wrist). Naively, I expect we need some sort of heuristic in Pathname.init(String) that detects if the Pathname was serialized on the other platform, then takes corrective action. Cases like 'c:/this/path' or '\\\\server\mount\point' serialized under Windows will have no exact meaning under non-Windows for which I don't know what would be reasonable behavior. Just ignore those parts?
The problem is that a pathname is serialized as its namestring. If the asdf code contains #P"/foo/bar" and it's compiled in Windows, that becomes #P"\\foo\\bar" and is serialized as such. What I did is to preserve the namestring passed by the user ("/foo/bar") unless of course the user later modifies the pathname, in which case the namestring is recomputed. Another possibility could be to store the user-provided string in another field (not the namestring) and use that field for serialization only. The field would be nulled when the pathname is modified and, if null, the namestring would be used for serialization. Probably the correct solution is to serialize a pathname as its distinct components and rebuild it from them, but it's significantly harder to do. Alessio
[Ticket #100 filed to track this issue][1]. [1]: http://trac.common-lisp.net/armedbear/ticket/100 -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Mon, Jun 14, 2010 at 2:10 PM, Mark Evenson <evenson.not.org@gmail.com> wrote:
[Ticket #100 filed to track this issue][1].
Thanks for officially tracking the problem. I think that "fixing" it only for asdf is simple: just emit (parse-namestring "...") instead of #P"...". That would allow to re-release 0.20 quickly. I can send a patch tonight for that. Eliminating the problem in the general case is harder and we'd better discuss the possible solutions. Here's my take at it: 1) don't fix it. State clearly that literal pathnames in source are OS-dependent. 2) keep, for each pathname constructed via #P or parse-namestring, the string passed by the user, until the pathname is destructively modified. When dumping it, use that string if available, else warn and use a calculated string. 3) dump-object for pathnames outputs a MAKE-PATHNAME form that reconstructs the pathname component by component. what do you think? Bye, Alessio
Hi, On Tue, Jun 15, 2010 at 2:05 PM, Alessio Stalla <alessiostalla@gmail.com> wrote:
On Mon, Jun 14, 2010 at 2:10 PM, Mark Evenson <evenson.not.org@gmail.com> wrote:
[Ticket #100 filed to track this issue][1].
Thanks for officially tracking the problem. I think that "fixing" it only for asdf is simple: just emit (parse-namestring "...") instead of #P"...". That would allow to re-release 0.20 quickly. I can send a patch tonight for that.
Eliminating the problem in the general case is harder and we'd better discuss the possible solutions. Here's my take at it:
1) don't fix it. State clearly that literal pathnames in source are OS-dependent.
2) keep, for each pathname constructed via #P or parse-namestring, the string passed by the user, until the pathname is destructively modified. When dumping it, use that string if available, else warn and use a calculated string.
3) dump-object for pathnames outputs a MAKE-PATHNAME form that reconstructs the pathname component by component.
I would definitely prefer (3), but that doesn't fix the general problem: what about objects which have pathnames as part of the serialization done by their MAKE-LOAD-FORM? [Or would dump-form also fix that?] Bye, Erik.
On Tue, Jun 15, 2010 at 3:21 PM, Erik Huelsmann <ehuels@gmail.com> wrote:
Hi,
On Tue, Jun 15, 2010 at 2:05 PM, Alessio Stalla <alessiostalla@gmail.com> wrote:
On Mon, Jun 14, 2010 at 2:10 PM, Mark Evenson <evenson.not.org@gmail.com> wrote:
[Ticket #100 filed to track this issue][1].
Thanks for officially tracking the problem. I think that "fixing" it only for asdf is simple: just emit (parse-namestring "...") instead of #P"...". That would allow to re-release 0.20 quickly. I can send a patch tonight for that.
Eliminating the problem in the general case is harder and we'd better discuss the possible solutions. Here's my take at it:
1) don't fix it. State clearly that literal pathnames in source are OS-dependent.
2) keep, for each pathname constructed via #P or parse-namestring, the string passed by the user, until the pathname is destructively modified. When dumping it, use that string if available, else warn and use a calculated string.
3) dump-object for pathnames outputs a MAKE-PATHNAME form that reconstructs the pathname component by component.
I would definitely prefer (3), but that doesn't fix the general problem: what about objects which have pathnames as part of the serialization done by their MAKE-LOAD-FORM? [Or would dump-form also fix that?]
MAKE-LOAD-FORM returns a form, not a string. How that form is translated to a string is another matter. I haven't looked at the compiler, but I see no obvious reason for not combining MAKE-LOAD-FORM with DUMP-FORM, if it's not already like that. Of course, the user - as opposed to ABCL itself - will need to be informed that simply PRINTing the load-form and later READing it on another OS might not work. Bye, Alessio
On 6/15/10 2:05 PM, Alessio Stalla wrote: […]
Eliminating the problem in the general case is harder and we'd better discuss the possible solutions. Here's my take at it:
1) don't fix it. State clearly that literal pathnames in source are OS-dependent.
2) keep, for each pathname constructed via #P or parse-namestring, the string passed by the user, until the pathname is destructively modified. When dumping it, use that string if available, else warn and use a calculated string.
I still don't understand how this approach solves the problem with ASDF. Wouldn't we still have the failure on non-Windows when "\foo\bar" is passed to the #P reader? Namely, that it would be interpreted as: { directory: nil, name: "\foo\bar" } What is the point of the "destructively modified" condition?
3) dump-object for pathnames outputs a MAKE-PATHNAME form that reconstructs the pathname component by component.
I was thinking about 4) Code a heuristic in the Pathname.init(String) that attempts to interpret a Pathname from the other platform by converting all '\' to '/'. Add something a) If the Pathname begins with '\\' i) if win interpret as UNC server ii) non-win signal a CONDITION b) convert all '\' to '/' c) run the rest of the current init() On win, the init() code (line 403++) will convert the '\' back to '/'. A problem arises for non-win Pathnames that contain true '\' characters in a directory/file name, which then will have no pathname<-->namestring roundtripping. It should still be possible to construct such shaggy monsters via MAKE-PATHNAME. As for load forms, I think simply marking the namestring as "transient" (i.e. not persisting it) will do the right thing as on first access it will be re-computed to the proper local conventions. This still doesn't completely address the presence of win-specific Pathname components (HOST with a UNC server, DEVICE with a single character string [A-Z]) in non-win. Signal a CONDITION when this is detected? N.B. All the above written without benefit of testing with ABCL, as my wrist still precludes heavy use of Emacs. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Thu, Jun 17, 2010 at 8:11 AM, Mark Evenson <evenson@panix.com> wrote:
On 6/15/10 2:05 PM, Alessio Stalla wrote:
[…]
Eliminating the problem in the general case is harder and we'd better discuss the possible solutions. Here's my take at it:
1) don't fix it. State clearly that literal pathnames in source are OS-dependent.
2) keep, for each pathname constructed via #P or parse-namestring, the string passed by the user, until the pathname is destructively modified. When dumping it, use that string if available, else warn and use a calculated string.
I still don't understand how this approach solves the problem with ASDF. Wouldn't we still have the failure on non-Windows when "\foo\bar" is passed to the #P reader? Namely, that it would be interpreted as:
{ directory: nil, name: "\foo\bar" }
What is the point of the "destructively modified" condition?
3) dump-object for pathnames outputs a MAKE-PATHNAME form that reconstructs the pathname component by component.
I was thinking about
4) Code a heuristic in the Pathname.init(String) that attempts to interpret a Pathname from the other platform by converting all '\' to '/'. Add something
a) If the Pathname begins with '\\' i) if win interpret as UNC server ii) non-win signal a CONDITION
b) convert all '\' to '/'
c) run the rest of the current init()
On win, the init() code (line 403++) will convert the '\' back to '/'.
A problem arises for non-win Pathnames that contain true '\' characters in a directory/file name, which then will have no pathname<-->namestring roundtripping. It should still be possible to construct such shaggy monsters via MAKE-PATHNAME.
As for load forms, I think simply marking the namestring as "transient" (i.e. not persisting it) will do the right thing as on first access it will be re-computed to the proper local conventions.
This still doesn't completely address the presence of win-specific Pathname components (HOST with a UNC server, DEVICE with a single character string [A-Z]) in non-win. Signal a CONDITION when this is detected?
N.B. All the above written without benefit of testing with ABCL, as my wrist still precludes heavy use of Emacs.
Yesterday Erik and I discussed this on IRC and we came up with a possible solution. Fact 1: the problem is only related to how pathnames are printed/serialized. Find a way to print them in a platform independent way (one which ABCL can read back, of course) and the problem is solved. Fact 2: the string representation of pathnames is completely implementation-dependent. Fact 3: pathnames already can (and sometimes do) print themselves as a property list enumerating the components, for example #P(:name "foo" :directory (:absolute "bar" "baz")) - which is not ANSI compatible, by the way. Proposed solution: let's invent an ABCL-specific way to print arbitrary pathnames. I proposed #P"abcl:(make-pathname ...)" which is ANSI-compatible and similar enough to what the current code in Pathname.writeToString can produce. Let's use that to print pathnames[1]. Reading them back is as simple as (eval (read-from-string ...)), and no other code needs to be modified. What do you think? [1] if not always, at least with dump-form, and when *print-readably* is T and the namestring can't be used. In the latter case currently the #P(...) syntax is used. Btw, probably dump-form *should* bind *print-readably* to T... Alessio
On 6/17/10 7:38 PM, Alessio Stalla wrote: […]
Proposed solution: let's invent an ABCL-specific way to print arbitrary pathnames. I proposed #P"abcl:(make-pathname ...)" which is ANSI-compatible and similar enough to what the current code in Pathname.writeToString can produce. Let's use that to print pathnames[1]. Reading them back is as simple as (eval (read-from-string ...)), and no other code needs to be modified.
What do you think?
[1] if not always, at least with dump-form, and when *print-readably* is T and the namestring can't be used. In the latter case currently the #P(...) syntax is used. Btw, probably dump-form *should* bind *print-readably* to T...
Hmmm, not sure I totally like if your proposal is that a PATHNAME is always in the #P"abcl:(make-pathname …)". If you are just proposing this is used when a namestring can't be produced, then I support this (weakly). I would claim that users are pretty cognitively wired at this point to expect a path be a string containing directory separators that we want to obey the "principle of least surprise" here. Your proposal still doesn't say what ABCL on non-Windows should do with a deserialized PATHNAME that represents a UNC path or has a drive letter in DEVICE. The current code (as I understand it) on non-Windows would treat a UNC pathname encoded as a string , e.g. "\\a\b\c\d", as an error as the '\c' doesn't represent a valid Java char escape sequence, although a case like "\\n\n\n\baz" would name the file with char sequence ('\' LF LF BS 'a' 'z'). Counter-proposal #1: Note that java.io.File *does* correctly accept "/" as a directory separator under Windows. So, we could potentially just declare "/" as our directory separator in the #P representation on all platforms. I would then make UNC pathnames not have a printable namestring, so inadvertent doubling of path separators doesn't cause confusion. For UNC and drive letters on non-Windows, I would signal a condition when a de-reference was attempted with a restart that tried to DWIM (ignore the UNC share, drop the drive letter reference, or allow a user supplied correction). Counter-proposal #2: Use URI (IRI?) for the namestring representation. This fits better to the nature of ABCL pathnames i.e. they aren't really just about filesystems at this point. '/' would again be the standard directory path separator. UNCs would get their own scheme (or we would enforce RFC 3986 so that 'file://server/share/dir/file' means UNC whereas 'file:///not/a/server/but/absolute' means an absolute pathname). Drive letters would be part of the URI authority ('file://c:/windows/path'). The same sort of DWIM condition/restarts for Windows-specific semantic on non-Windows would be available. As a user convenience, we might make the 'file://' prefix be optionally inferred. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Fri, Jun 18, 2010 at 12:37 PM, Mark Evenson <evenson@panix.com> wrote:
On 6/17/10 7:38 PM, Alessio Stalla wrote: […]
Proposed solution: let's invent an ABCL-specific way to print arbitrary pathnames. I proposed #P"abcl:(make-pathname ...)" which is ANSI-compatible and similar enough to what the current code in Pathname.writeToString can produce. Let's use that to print pathnames[1]. Reading them back is as simple as (eval (read-from-string ...)), and no other code needs to be modified.
What do you think?
[1] if not always, at least with dump-form, and when *print-readably* is T and the namestring can't be used. In the latter case currently the #P(...) syntax is used. Btw, probably dump-form *should* bind *print-readably* to T...
Hmmm, not sure I totally like if your proposal is that a PATHNAME is always in the #P"abcl:(make-pathname …)". If you are just proposing this is used when a namestring can't be produced, then I support this (weakly).
No, this is precisely the problem we have today. On Windows, for #P"/foo/bar" a namestring *can* be produced, but it is "\foo\bar", which then can't be read back in on non-windows.
I would claim that users are pretty cognitively wired at this point to expect a path be a string containing directory separators that we want to obey the "principle of least surprise" here.
In fact, I'm in doubt whether to always use the ugly form or only when serializing to fasls and when today we would have used #P(...)
Your proposal still doesn't say what ABCL on non-Windows should do with a deserialized PATHNAME that represents a UNC path or has a drive letter in DEVICE.
What could it possibly do but fail if you try to OPEN the pathname? The problem is not Windows pathnames on non-Windows. Those cannot work and it's the user's responsibility to use them only if she knows that the software will only run on Windows. The problem are Unix pathnames that, when printed under Windows, have their slashes converted to backslashes.
The current code (as I understand it) on non-Windows would treat a UNC pathname encoded as a string , e.g. "\\a\b\c\d", as an error as the '\c' doesn't represent a valid Java char escape sequence, although a case like "\\n\n\n\baz" would name the file with char sequence ('\' LF LF BS 'a' 'z').
That's the same under Windows. String escaping has nothing to do with pathnames.
Counter-proposal #1: Note that java.io.File *does* correctly accept "/" as a directory separator under Windows. So, we could potentially just declare "/" as our directory separator in the #P representation on all platforms. I would then make UNC pathnames not have a printable namestring, so inadvertent doubling of path separators doesn't cause confusion. For UNC and drive letters on non-Windows, I would signal a condition when a de-reference was attempted with a restart that tried to DWIM (ignore the UNC share, drop the drive letter reference, or allow a user supplied correction).
I wouldn't do this DWIM thing. A UNC or drive-letter pathname on non-windows is a user error, imho. As for always using / as separator, it would solve the problem with asdf, but it wouldn't solve the other problem that not all pathnames are currently printable by abcl as #P"..." and so it sometimes uses #P(...) to list their components, which is not ANSI-compatible.
Counter-proposal #2: Use URI (IRI?) for the namestring representation. This fits better to the nature of ABCL pathnames i.e. they aren't really just about filesystems at this point. '/' would again be the standard directory path separator. UNCs would get their own scheme (or we would enforce RFC 3986 so that 'file://server/share/dir/file' means UNC whereas 'file:///not/a/server/but/absolute' means an absolute pathname). Drive letters would be part of the URI authority ('file://c:/windows/path'). The same sort of DWIM condition/restarts for Windows-specific semantic on non-Windows would be available. As a user convenience, we might make the 'file://' prefix be optionally inferred.
The problem with URIs is that they cannot represent all of CL's pathname components (version?). We'd need to invent an encoding. My proposal is simpler because an encoding is basically already there. However, I'm not against using URIs with an appropriate encoding. Bye, Alessio
On 6/18/10 1:00 PM, Alessio Stalla wrote:
On Fri, Jun 18, 2010 at 12:37 PM, Mark Evenson<evenson@panix.com> wrote:
On 6/17/10 7:38 PM, Alessio Stalla wrote:
[…]
Hmmm, not sure I totally like if your proposal is that a PATHNAME is always in the #P"abcl:(make-pathname …)". If you are just proposing this is used when a namestring can't be produced, then I support this (weakly).
No, this is precisely the problem we have today. On Windows, for #P"/foo/bar" a namestring *can* be produced, but it is "\foo\bar", which then can't be read back in on non-windows.
But since we control the production of the namestring, I am proposing we only ever consider a string with '/' as the directory path separator. '\' is banished. We provide a convenience function to convert PATHNAME to a Windows "native" representation. […]
Your proposal still doesn't say what ABCL on non-Windows should do with a deserialized PATHNAME that represents a UNC path or has a drive letter in DEVICE.
What could it possibly do but fail if you try to OPEN the pathname?
It's a bit more subtle than that: Pathname.java contains runtime conditionals for win/non-win platform meaning that constructing/parsing PATHNAMEs work differently on each platform. I am a little worried about MERGE-PATHNAME-DEFAULTS without a default DEVICE inserting the current drive letter for DEVICE under Windows. Maybe this isn't a big issue.
The problem is not Windows pathnames on non-Windows. Those cannot work and it's the user's responsibility to use them only if she knows that the software will only run on Windows. The problem are Unix pathnames that, when printed under Windows, have their slashes converted to backslashes.
The current code (as I understand it) on non-Windows would treat a UNC pathname encoded as a string , e.g. "\\a\b\c\d", as an error as the '\c' doesn't represent a valid Java char escape sequence, although a case like "\\n\n\n\baz" would name the file with char sequence ('\' LF LF BS 'a' 'z').
That's the same under Windows. String escaping has nothing to do with pathnames.
No it isn't the same, as under Windows, Pathname.init(String) will convert this to a PATHNAME that uses HOST to store the UNC server and share name. Under non-Windows, the eventual call to File(String) will have to contend with what looks to Java like backslash escapes as "\\a\b\c\d" is stored in the NAME field.
Counter-proposal #1: Note that java.io.File *does* correctly accept "/" as a directory separator under Windows. So, we could potentially just declare "/" as our directory separator in the #P representation on all platforms. I would then make UNC pathnames not have a printable namestring, so inadvertent doubling of path separators doesn't cause confusion. For UNC and drive letters on non-Windows, I would signal a condition when a de-reference was attempted with a restart that tried to DWIM (ignore the UNC share, drop the drive letter reference, or allow a user supplied correction).
I wouldn't do this DWIM thing. A UNC or drive-letter pathname on non-windows is a user error, imho. As for always using / as separator, it would solve the problem with asdf, but it wouldn't solve the other problem that not all pathnames are currently printable by abcl as #P"..." and so it sometimes uses #P(...) to list their components, which is not ANSI-compatible.
It's a user error certainly, but if it is a correctable situation providing a restart is a pretty decent way to go. Such a user error may easily be occurring in an ASDF package that would otherwise work fine under ABCL, which I would like to allow. That not all pathnames are printable should be corrected in an ANSI-compatible manner as you describe. This is a orthogonal to the backslash/forward slash problems, right? This apparently confused me until your reply here.
Counter-proposal #2: Use URI (IRI?) for the namestring representation. This fits better to the nature of ABCL pathnames i.e. they aren't really just about filesystems at this point. '/' would again be the standard directory path separator. UNCs would get their own scheme (or we would enforce RFC 3986 so that 'file://server/share/dir/file' means UNC whereas 'file:///not/a/server/but/absolute' means an absolute pathname). Drive letters would be part of the URI authority ('file://c:/windows/path'). The same sort of DWIM condition/restarts for Windows-specific semantic on non-Windows would be available. As a user convenience, we might make the 'file://' prefix be optionally inferred.
The problem with URIs is that they cannot represent all of CL's pathname components (version?). We'd need to invent an encoding. My proposal is simpler because an encoding is basically already there. However, I'm not against using URIs with an appropriate encoding.
I amend my second proposal to "all Pathnames that have a straightforward URI representation use it; others use the ANSI compatible mechanism proposed by Alessio". But don't consider #2 for right now: think about #1: "no use of '\' as a directory separator in ABCL namestrings". -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
Attached is a patch that (partially) normalizes the Pathname namestring across all platforms to only contain '/' characters as the directory separators, which is my preferred way to solve the portability problem for pathname serialization. Feeding '\' separated paths to Pathname still seems to work. This work needs: 1. Agreement from others (Alessio, Erik?) that this is the "right" way to solve the problem 2. The MAKE-LOAD-FORM seems to be dumping the Windows version of the path, which means the code is not finished. 3. A convenience function to return the '\' separated form for use under Windows. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
The (revised, tested, working) attached patch solves the cross-platform Pathname serialization problem by eliminating the output of '\' characters in namestrings as they are all now interpreted and converted to '/' regardless of platform. 1. UNC paths now have problems. My proposal would be that UNC paths always use the "unprintable namestring" representation when I figure out what that should be like 2. No convenience function to convert a namestring to Windows '\' forms. Since Windows accepts '/' as a directory separator is this really necessary? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On 6/17/10 7:38 PM, Alessio Stalla wrote: […]
Fact 3: pathnames already can (and sometimes do) print themselves as a property list enumerating the components, for example #P(:name "foo" :directory (:absolute "bar" "baz")) - which is not ANSI compatible, by the way.
Proposed solution: let's invent an ABCL-specific way to print arbitrary pathnames. I proposed #P"abcl:(make-pathname ...)" which is ANSI-compatible and similar enough to what the current code in Pathname.writeToString can produce. Let's use that to print pathnames[1]. Reading them back is as simple as (eval (read-from-string ...)), and no other code needs to be modified.
I can't easily find a reference to figure out what is non-ANSI about our current non-printable namestring representation. Could you provide a pointer and/or a summary to what constraints we are operating under here? Or maybe it has to do more with reader macros than namestrings? -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
The PATHNAME serialization and ANSI-compatible namestrings issues should be solved as of [svn r12810][1]. 1. A PATHNAME with no namestring now has a non-printable representation (i.e. "#<PATHNAME (no namestring) …"). 2. The old, non-ANSI compatible #P(:host "host" :device nil …) output has been removed. 3. A PATHNAME namestring only contains '/' characters as directory separators. Any '\' passed in are converted. 4. UNC pathnames can be constructed via '\\server\mount\path' but have a non-printable namestring representations. Hopefully this puts these issues to rest. Any other outstanding problems related to this issues should be reported as bugs. [1]: http://trac.common-lisp.net/armedbear/changeset/12810 -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
participants (5)
-
Alessio Stalla -
Erik Huelsmann -
Knut Olav Bøhmer -
Mark Evenson -
Mark Evenson