Dear ASDF developers,

there were test failures recently in SCL, and Douglas Crosher found that they were related to the semantics of merge-pathnames*. I'm not sure anymore what merge-pathnames does, how its current behavior differs from SCL's ext:resolve-pathname, and more importantly, what it should be doing, so I'm thinking aloud and maybe one of you will catch something.

...

...

: Douglas Crosher

: Fare

: Douglas Crosher

...

...

The SCL requires slightly different code in merge-pathnames* which seems to have been causing the issue.

The issue occurs because pathnames in the test do not have a host etc so are resolved relative to a logical-pathname host:

(merge-pathnames* "/home/mod1/file.lisp" #p"ASDFTEST:SRC;") =>  #p"ASDFTEST:HOME;MOD1;FILE.LISP.NEWEST"

Adding the host etc gives the following and corrects the issue:

(merge-pathnames* "file://localhost/home/mod1/file.lisp" #p"ASDFTEST:SRC;") =>  #p"file://localhost/home/mod1/file.lisp"

I can't see any reason why a pathname with an absolute directory should be handled specially in merge-pathnames* and the following example seems right for the SCL:

(merge-pathnames* "/home/mod1/file.lisp" #p"http://host/") =>  #p"http://host/home/mod1/file.lisp"

The reason why merge-pathnames* does this is because we want to treat the merging of relative pathnames specially: based on defsystem specifications, we have pathnames that are relative to wherever the .asd file is; however, at the time the user specifies those relative pathnames, and as he does some merging operations on them, he has no idea what kind of implementation will be present and what host (logical or physical), device, etc., will be used for the pathname. Therefore, I needed this abstraction (and accompanying syntax) for "pathnames wherein a relative pathname does not specify host or device, but an absolute pathname does".

I suppose there's a slight mismatch with scl's ext:resolve-pathname. Instead of your first patch, maybe merge-pathnames* shouldn't just call that latter function, but do some more wrapping around it?

The SCL's ext:resolve-pathname seems to handle the relative paths just fine, and works with URLs.

The problem occurred when the system definition specified a path with an absolute directory and no host or device. The SCL's ext:resolve-pathname will resolve such a path relative to the base host etc which I think is useful.

Consider a standard CL example of a system definition with a base pathname which is a logical pathname.  In this case it would be reasonable to have a component pathname with an absolute directory and no host be resolved relative to the base logical pathname - which is what the SCL's ext:resolve-pathname will do.  The current behaviour of merge-pathname* is to simply ignore the default host and device for a pathname with an absolute directory which does not seem useful.  Perhaps you could consider modifying merge-pathname* to merge the host and device for pathname with an absolute directory rather than ignoring the defaults in this case.

This issue is hard to describe, but I may be able to come up with a new test for it.

OK. I think I get it.

The issue is that at least *some* CL implementations (for instance clisp) use a NIL pathname-host for normal Unix absolute pathnames. In that case, the host being NIL doesn't mean "host unspecified", but "normal absolute pathname", and it is wrong to replace it with a say logical host from the defaults.

Maybe the "right thing" then is to identify, in each implementation, what is the canonical way of representing the possible "absolute pathname referring to some default toplevel filesystem" as opposed to possibly "absolute pathname depending on unspecified host", and keep a NIL in the first case (where that applies) but not the other case. Then, the asdf-pathname-test would have to be updated to do the right thing with those two cases (whichever applies). It seems that your ext:resolve-pathnames might already be doing this "right thing" — except maybe when a relative pathname is specified with a host?

NB: to maximize the chances of pathnames being created with the correct host and device, maybe load-sysdef should bind *default-pathname-defaults* to (pathname-directory-pathname pathname) around load?

Oh noes, more portability issues with CL pathnames!

At that point, I'll just weep.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org