[asdf-devel] never ending component relative pathnames
good evening; on the occasion of pushing de.setf.graphics down the wire, when i built it for ccl/sbcl i did an obligatory pull on asdf and observe that something changed in the treatment of modules' component relative pathnames. with the effect that it was no longer possible possible to build clx. the clx-0-7-4 version as (:relative) specification in module pathnames which ran afoul of the asdf changes. i applied the same tactic as i have previously found effective for source file components and was able to build. the diff [1] is posted with the graphics sources. on other semi-related matters, i can report, that i have now an s3 ami (linux-2.6.31+ubuntu++^3) with which i can boot an ec2 instance with all pieces in place to run the target lisp implementations. --- [1] : http://github.com/lisp/de.setf.graphics/blob/master/readmes/ asdf.diff
On 3/5/10 Mar 5 -12:06 PM, james anderson wrote:
good evening;
on the occasion of pushing de.setf.graphics down the wire, when i built it for ccl/sbcl i did an obligatory pull on asdf and observe that something changed in the treatment of modules' component relative pathnames. with the effect that it was no longer possible possible to build clx. the clx-0-7-4 version as (:relative) specification in module pathnames which ran afoul of the asdf changes. i applied the same tactic as i have previously found effective for source file components and was able to build. the diff [1] is posted with the graphics sources.
on other semi-related matters, i can report, that i have now an s3 ami (linux-2.6.31+ubuntu++^3) with which i can boot an ec2 instance with all pieces in place to run the target lisp implementations.
--- [1] : http://github.com/lisp/de.setf.graphics/blob/master/readmes/ asdf.diff
I reviewed this modification and I'm not sure I understand the implications. This seems to squash Fare's component-name-to-pathname-components call, and I don't know what implications that has for his newfangled component names like (:file "foo/bar"). [As an aside, do we have tests for these? I see them only in test-module-pathnames and only in one location...] I'm reluctant to apply this patch without more understanding of its effects. Best, r
i would be as well. that's why i sent it to you at the other end of a link. in any case, whatever direction the component location computations follow, they need somehow to take into account the variations in pathname construction which ccl/sbcl evidence. On 2010-03-05, at 20:14 , Robert Goldman wrote:
On 3/5/10 Mar 5 -12:06 PM, james anderson wrote:
good evening;
on the occasion of pushing de.setf.graphics down the wire, when i built it for ccl/sbcl i did an obligatory pull on asdf and observe that something changed in the treatment of modules' component relative pathnames. with the effect that it was no longer possible possible to build clx. the clx-0-7-4 version as (:relative) specification in module pathnames which ran afoul of the asdf changes. i applied the same tactic as i have previously found effective for source file components and was able to build. the diff [1] is posted with the graphics sources.
on other semi-related matters, i can report, that i have now an s3 ami (linux-2.6.31+ubuntu++^3) with which i can boot an ec2 instance with all pieces in place to run the target lisp implementations.
--- [1] : http://github.com/lisp/de.setf.graphics/blob/master/readmes/ asdf.diff
I reviewed this modification and I'm not sure I understand the implications. This seems to squash Fare's component-name-to-pathname-components call, and I don't know what implications that has for his newfangled component names like (:file "foo/bar"). [As an aside, do we have tests for these? I see them only in test-module-pathnames and only in one location...]
I'm reluctant to apply this patch without more understanding of its effects.
Best, r
On 3/5/10 Mar 5 -2:18 PM, james anderson wrote:
i would be as well.
that's why i sent it to you at the other end of a link.
I followed the link to the diff. What more should I have seen? Also, it would be helpful if you would describe precisely the breakage you have observed (what precisely happened to the module pathnames?). In that case, we can abstract a simpler version and turn it into a test case. We can't reverse engineer such a test case out of the description below, and it's probably expecting too much for us to pull your git repo and try to build it just in order to extract a bug. Would it be possible for you to reformulate this as a launchpad bug? I'm afraid I'm very busy with paying work now, and won't be able to do much more than push a very clear patch for the next week or so. Maybe someone else has more time. Best, R
in any case, whatever direction the component location computations follow, they need somehow to take into account the variations in pathname construction which ccl/sbcl evidence.
On 2010-03-05, at 20:14 , Robert Goldman wrote:
On 3/5/10 Mar 5 -12:06 PM, james anderson wrote:
good evening;
on the occasion of pushing de.setf.graphics down the wire, when i built it for ccl/sbcl i did an obligatory pull on asdf and observe that something changed in the treatment of modules' component relative pathnames. with the effect that it was no longer possible possible to build clx. the clx-0-7-4 version as (:relative) specification in module pathnames which ran afoul of the asdf changes. i applied the same tactic as i have previously found effective for source file components and was able to build. the diff [1] is posted with the graphics sources.
on other semi-related matters, i can report, that i have now an s3 ami (linux-2.6.31+ubuntu++^3) with which i can boot an ec2 instance with all pieces in place to run the target lisp implementations.
--- [1] : http://github.com/lisp/de.setf.graphics/blob/master/readmes/ asdf.diff
I reviewed this modification and I'm not sure I understand the implications. This seems to squash Fare's component-name-to-pathname-components call, and I don't know what implications that has for his newfangled component names like (:file "foo/bar"). [As an aside, do we have tests for these? I see them only in test-module-pathnames and only in one location...]
I'm reluctant to apply this patch without more understanding of its effects.
Best, r
_______________________________________________ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
good afternoon; On 2010-03-05, at 21:35 , Robert Goldman wrote:
On 3/5/10 Mar 5 -2:18 PM, james anderson wrote:
i would be as well.
that's why i sent it to you at the other end of a link.
I followed the link to the diff. What more should I have seen?
Also, it would be helpful if you would describe precisely the breakage you have observed (what precisely happened to the module pathnames?).
In that case, we can abstract a simpler version and turn it into a test case. We can't reverse engineer such a test case out of the description below, and it's probably expecting too much for us to pull your git repo and try to build it just in order to extract a bug.
Would it be possible for you to reformulate this as a launchpad bug?
please find enclosed suggestion as to the use cases which the component pathname computation should support. it is formulated as a test, whether the component pathname for each of the system, module, and source file component in a minimal system definition does in fact locate the intended directory or file given combinations of component name and :pathname argument in the respective system declaration. for each combination this tests the system directory pathname, module directory pathname the actual source file pathname and that pathname with an explicit type default. i attach also the results for sbcl and ccl based on asdf >= 1.630 in all there are more than a thousand combinations. of these i observe two systematic failures. - if a logical pathname is specified as a string, it is not correctly coerced to a logical pathname. in sbcl it causes an error. in ccl the pathname is wrong. - if a relative pathname is specified for a source file, either the file name is missing in the component pathname or the type is missing depending on the declaration. there may be some combinations for which the expected target is wrong - with the consequent false positive/negative. if one could eliminate the systematic failures, should any still remain, we can look again.
If this gets loaded into the asdf git repo, I'll test on ACL linux and Mac. Might manage windows, too. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
good afternoon; On 2010-03-11, at 15:37 , Robert Goldman wrote:
If this gets loaded into the asdf git repo, I'll test on ACL linux and Mac. Might manage windows, too.
it's in the asdf-x repo. in the test folder.[1] the link in my earlier message was to a directory with results from the full complement of linux runtimes. if you want to see them, i can bring it back on-line. otherwise i'll wait until i've worked in the changes from the previous several days of messages. --- [1] : http://github.com/lisp/de.setf.asdf.x/tree/master/test/
On 3/11/10 Mar 11 -8:51 AM, james anderson wrote:
good afternoon;
On 2010-03-11, at 15:37 , Robert Goldman wrote:
If this gets loaded into the asdf git repo, I'll test on ACL linux and Mac. Might manage windows, too.
it's in the asdf-x repo. in the test folder.[1] the link in my earlier message was to a directory with results from the full complement of linux runtimes. if you want to see them, i can bring it back on-line. otherwise i'll wait until i've worked in the changes from the previous several days of messages.
--- [1] : http://github.com/lisp/de.setf.asdf.x/tree/master/test/
Got it. I'll do this when it's in the ASDF repo. thanks, r
good evening; On 2010-03-11, at 15:37 , Robert Goldman wrote:
If this gets loaded into the asdf git repo, I'll test on ACL linux and Mac. Might manage windows, too
not yet in the asdf repo, but now reformulated to follow faré's advise in recent correspondence: http://ec2-174-129-136-72.compute-1.amazonaws.com/test/ 20100314T154830/ the results indicate progress. some implementations still show errors in system construction, some do not find all intended pathnames, but ccl and sbcl are well improved. as long as the :pathname argument is always a pathname. if it is a string all bets are off.
good evening; here is yet another update.[0] where no string values are permitted for :pathname arguments - not even those which i had understood should be supported, the results are almost uniform. 20100314T224221/alisp/output.txt: (:RESULT :TYPE "International Allegro CL Free Express Edition" :VERSION "8.2 [Linux (x86)] (Jan 25, 2010 14:49)" :FILE #P"output.txt" :SYSTEM-FAILURES (0 320) :DIRECTORY-FAILURES (0 1280) :FILE-FAILURES (213 2080) :HOMOGENEOUS 85) 20100314T224221/ccl/output.txt: (:RESULT :TYPE "Clozure Common Lisp" :VERSION "Version 1.4-r13119 (LinuxX8632)" :FILE #P"output.txt" :SYSTEM-FAILURES (0 320) :DIRECTORY-FAILURES (0 1280) :FILE-FAILURES (0 2080) :HOMOGENEOUS 0) 20100314T224221/clisp/output.txt: (:RESULT :TYPE "CLISP" :VERSION "2.48+ (2009-08-24) (built on ip-10-251-121-194.ec2.internal [10.251.121.194])" :FILE #P"output.txt" :SYSTEM-FAILURES (148 327) :DIRECTORY-FAILURES (0 1197) :FILE-FAILURES (43 2378) :HOMOGENEOUS 35) 20100314T224221/cmucl/output.txt: (:RESULT :TYPE "CMU Common Lisp" :VERSION "20a (20A)" :FILE #P"output.txt" :SYSTEM-FAILURES (0 320) :DIRECTORY-FAILURES (0 1280) :FILE-FAILURES (0 2080) :HOMOGENEOUS 0) 20100314T224221/ecl/output.txt: (:RESULT :TYPE "ECL" :VERSION "10.2.1" :FILE #P"output.txt" :SYSTEM-FAILURES (0 320) :DIRECTORY-FAILURES (0 1280) :FILE-FAILURES (0 2080) :HOMOGENEOUS 0) 20100314T224221/lw/output.txt: (:RESULT :TYPE "LispWorks Personal Edition" :VERSION "5.1.1" :FILE #P"/ebs/test/20100314T224221/lw/output.txt" :SYSTEM-FAILURES (0 320) :DIRECTORY-FAILURES (0 1280) :FILE-FAILURES (0 2080) :HOMOGENEOUS 0) 20100314T224221/sbcl/output.txt: (:RESULT :TYPE "SBCL" :VERSION "1.0.36" :FILE #P"/ebs/test/ 20100314T224221/sbcl/output.txt" :SYSTEM-FAILURES (0 320) :DIRECTORY-FAILURES (0 1280) :FILE-FAILURES (0 2080) :HOMOGENEOUS 0) - the pathname merge operator must be modified[1] to be strict with the arguments it supplies to make-pathname - allegro[3] fails for 213 file components. it occasionally constructs an unintended pathname by supplying a spurious type - clisp[4] fails to construct 148 systems and fails to find 43 files. it occasionally constructs an unintended pathname by incorporating a logical pathname's namestring as a file name --- [0] : http://ec2-174-129-136-72.compute-1.amazonaws.com/test/ 20100314T224221.txt [1] : http://ec2-174-129-136-72.compute-1.amazonaws.com/test/asdf- diff.txt [3] : http://ec2-174-129-136-72.compute-1.amazonaws.com/test/ 20100314T224221/alisp/output.txt [4] : http://ec2-174-129-136-72.compute-1.amazonaws.com/test/ 20100314T224221/clisp/output.txt asdf.lisp diff: diff --git a/asdf.lisp b/asdf.lisp index 7858caa..470cbda 100644 --- a/asdf.lisp +++ b/asdf.lisp @@ -513,8 +513,13 @@ does not have an absolute directory, then the HOST and DEVICE come from the DEFA (values (pathname-host defaults) (pathname-device defaults) (append (pathname-directory defaults) (cdr directory))))) - (make-pathname :host host :device device :directory directory - :name name :type type :version version)))) + + (apply #'make-pathname :host host :device device :directory directory + `(,@(case version ((nil :unspecific) nil) (t `(:version ,version))) + ,@(case name ((nil :unspecific) nil) (t `(:name ,name))) + ,@(case type ((nil :unspecific) nil) (t `(:type ,type))))) + #+(or) (make-pathname :host host :device device :directory directory + :name name :type type :version version)))) (define-modify-macro appendf (&rest args) append "Append onto list")
Dear James, is there a tarball I can download with enough stuff to replace those tests or yours? I don't understand which cases exactly fail on which implementations. Also, why do you map :unspecific to nil in merge-pathnames* ? What issues does that solve? Isn't that against what we want? Shouldn't we rather have split-name-type return type :unspecific more often? Maybe your function should compare namestrings instead of pathnames? Thanks again for your testing and your forcing us to do things right. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Every country has an army, either its own or a foreign one.
On 2010-03-15, at 01:36 , Faré wrote:
Dear James,
is there a tarball I can download with enough stuff to replace those tests or yours?
? the tests are all in the one asdf-pathname-test.lisp file.
I don't understand which cases exactly fail on which implementations.
the respective output.txt file lists the failures.[0] the head of the file has three lists: - the pathnames which i understand to be the complete set given the enumerated variations. - the logical host mapping - the system/module/file pathname variations
if the system definition failed, the definition is cited with the error message. if the pathname reference failed, the entry looks like CL-SOURCE-FILE "file" missing: #P"/ebs/test/asdf-src/system1/module1/ untyped-file.lisp" configuration: (#P"ASDFTEST:system1;" #P"ASDFTEST:system2;module4;" #P"/ebs/test/asdf-src/system1/module1/ untyped-file") parent pathnames: (#P"ASDFTEST:system1;" #P"ASDFTEST:system2;module4;") this displays <the component type> <the component name> missing: <the actual component pathname> which is a (physical . logical) pair if that applies configuration: (<system pathname arg> <module pathname arg> <file pathname arg>) parent pathnames: (<actual system pathname> <actual module pathname>) the last line in each file reports the counts
Also, why do you map :unspecific to nil in merge-pathnames* ? What issues does that solve? Isn't that against what we want?
i changed it to pass no argument when no value is intended. on this topic clhs/make-pathname -> 19.2.1 lets you down. you need to attend to 19.3.2.1 [1], to which it appears lispworks pays attention.[2]
Shouldn't we rather have split-name-type return type :unspecific more often? Maybe your function should compare namestrings instead of pathnames?
the test does not explicitly "compare". it create the files, then uses the component pathnames to try to write to them (or just probe for directories), and then confirms that it found all the intended files by checking their modification times. at least, that is what it is supposed to do. --- [0] : http://ec2-174-129-136-72.compute-1.amazonaws.com/test/ 20100314T224221/ [1] : http://www.lispworks.com/documentation/HyperSpec/Body/19_cba.htm [2] : http://ec2-174-129-136-72.compute-1.amazonaws.com/test/ 20100309T173540/lw/output.txt
good afternoon; [the first version of this message tripped on the size threshold for attachments. this copy is with links] On 2010-03-05, at 21:35 , Robert Goldman wrote:
On 3/5/10 Mar 5 -2:18 PM, james anderson wrote:
i would be as well.
that's why i sent it to you at the other end of a link.
I followed the link to the diff. What more should I have seen?
Also, it would be helpful if you would describe precisely the breakage you have observed (what precisely happened to the module pathnames?).
In that case, we can abstract a simpler version and turn it into a test case. We can't reverse engineer such a test case out of the description below, and it's probably expecting too much for us to pull your git repo and try to build it just in order to extract a bug.
Would it be possible for you to reformulate this as a launchpad bug?
please find referenced below, a suggestion as to the use cases which the component pathname computation should support.[1] it is formulated as a test, whether the component pathname for each of the system, module, and source file component in a minimal system definition does in fact locate the intended directory or file given combinations of component name and :pathname argument in the respective system declaration. for each combination this tests the system directory pathname, module directory pathname the actual source file pathname and that pathname with an explicit type default. i attach also the results for sbcl[2] and ccl[3] based on asdf >= 1.630. in all there are more than a thousand combinations. of these i observe two systematic failures. - if a logical pathname is specified as a string, it is not correctly coerced to a logical pathname. in sbcl it causes an error. in ccl the pathname is wrong. - if a relative pathname is specified for a source file, either the file name is missing in the component pathname or the type is missing depending on the declaration. there may be some combinations for which the expected target is wrong - with the consequent false positive/negative. if one could eliminate the systematic failures, should any still remain, we can look again. --- [1] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp- test.lisp [2] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp- test-results-fasl.txt [3] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp- test-results-dfsl.txt
Dear James, On 8 March 2010 11:08, james anderson <james.anderson@setf.de> wrote:
please find referenced below, a suggestion as to the use cases which the component pathname computation should support.[1]
specifying :pathname arguments to ASDF components as strings had NEVER been working portably before ASDF 1.630. The current behavior might not fit all the non-portable expectations that someone or some other might have fancied at some point, but 1- it is now portable, and 2- any previous non-portable expectation can still achieved with a simple #p prefix to your namestring. Therefore, I think the current behavior is 100% better than the previous one. It indeed needs to be better documented, though. It would be nice if from your file was extracted a test for the actually supported combinations. I'd include that in the ASDF test suite. As for the non-supported combinations, well, they are not supported, never have been, and probably never will. Note interestingly that #p"..." pathnames other than logical pathnames are not portable either, are not supported, and should NOT be tested in the test-suite.
- if a logical pathname is specified as a string, it is not correctly coerced to a logical pathname. in sbcl it causes an error. in ccl the pathname is wrong. Not supported. Never will be. Please use #p for that.
- if a relative pathname is specified for a source file, either the file name is missing in the component pathname or the type is missing depending on the declaration. Can you give an example?
there may be some combinations for which the expected target is wrong - with the consequent false positive/negative. if one could eliminate the systematic failures, should any still remain, we can look again. If you can produce a list of failures in cases that should be supported, that'd be great.
[1] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp-test.lisp [2] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp-test-results-fasl.... [3] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp-test-results-dfsl....
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] It is useless to attempt to reason a man out of a thing he was never reasoned into. — Jonathan Swift
good evening; On 2010-03-08, at 17:53 , Faré wrote:
Dear James,
On 8 March 2010 11:08, james anderson <james.anderson@setf.de> wrote:
please find referenced below, a suggestion as to the use cases which the component pathname computation should support.[1]
specifying :pathname arguments to ASDF components as strings had NEVER been working portably before ASDF 1.630.
i understand, that in some environments neither logical pathname designators nor logical pathnames themselves are seen as portable, but i continue to try to treat them as if they were. so far, with success. mostly.
The current behavior might not fit all the non-portable expectations that someone or some other might have fancied at some point, but 1- it is now portable, and 2- any previous non-portable expectation can still achieved with a simple #p prefix to your namestring.
that adjustment to the problematic .asd had, in fact, brought it in line with the changed asdf behaviour. if asdf was never intended to support such designators, that's also ok. r. goldman had requested that i describe the breakage i had observed after i had pulled the then new asdf version. that is what this enumeration provided.
Therefore, I think the current behavior is 100% better than the previous one. It indeed needs to be better documented, though.
that would be nice.
It would be nice if from your file was extracted a test for the actually supported combinations. I'd include that in the ASDF test suite. As for the non-supported combinations, well, they are not supported, never have been, and probably never will.
Note interestingly that #p"..." pathnames other than logical pathnames are not portable either, are not supported, and should NOT be tested in the test-suite.
i did not intend for there to have been any. i did not think that there were.
- if a logical pathname is specified as a string, it is not correctly coerced to a logical pathname. in sbcl it causes an error. in ccl the pathname is wrong. Not supported. Never will be. Please use #p for that.
- if a relative pathname is specified for a source file, either the file name is missing in the component pathname or the type is missing depending on the declaration. Can you give an example?
there may be some combinations for which the expected target is wrong - with the consequent false positive/negative. if one could eliminate the systematic failures, should any still remain, we can look again. If you can produce a list of failures in cases that should be supported, that'd be great.
if someone would enumerate the cases which are supported. i will try again. as i understand your demurral, relative to the tested enumeration[2], one should eliminate the cases which involve string designators for logical pathnames. should anything else be removed? are that any additional cases for which support should be tested? --- [1] : http://github.com/lisp/de.setf.graphics/blob/master/graphics.asd [2] : http://github.com/lisp/de.setf.asdf.x/blob/master/test/cp- test.lisp
i understand, that in some environments neither logical pathname designators nor logical pathnames themselves are seen as portable, but i continue to try to treat them as if they were. so far, with success. mostly.
I believe logical pathnames as defined in the CLHS are widely supported, and if any implementation has bugs, then the implementation should be fixed, not ASDF. However, they also have limitations, and if you rely on more than what the specification provides, you're on your own unless and until you get the spec fixed.
if asdf was never intended to support such designators, that's also ok. It was never. The only previously supported way that string could be used as pathname designators was as file or directory names without any directory separator, host or device specification. Now you can also portably use / as a directory separator, but there is no way to specify host or device or absolute pathname. Use #p... or #.(make-pathname ...) for that.
r. goldman had requested that i describe the breakage i had observed after i had pulled the then new asdf version. that is what this enumeration provided. Thanks a lot, it is useful. But a distinction should be made between supported cases and non-supported cases.
Now supported (1.630): 1- the name is a string or symbol (the name of which will be downcased). It will be interpreted by merge-component-name-type as a /-separated path, using the type specified by the component class if any. 2- the :pathname specification overrides the name. It is interpreted similarly if a symbol or a string. It can also be a pathname, in which case it is not further interpreted. 3- the parent's pathname from is an absolute pathname, with proper host and device. 4- the pathname obtained from 1 or 2 is merged with the one from 3 as with merge-pathnames*, i.e. if it's relative, then the parent's host and device are used. Actually, now that we are using merge-pathnames* instead of merge-pathnames, I find that the default argument to merge-component-name-type becomes useless. Unless anyone objects, I will remove it in ASDF 1.631 (and rename the function). Regarding 2, note that the only portable ways to obtain a pathname are a- #p"LOGICAL-HOST:logical;path;name" b- #.(make-pathname ...) c- More #. magic with merge-pathnames, *load-pathname*, *load-truename*, (user-homedir-pathname), etc. In particular, non-logical namestrings are NOT portable, MUST NOT be used in a portable .asd file, and are otherwise not supported.
Therefore, I think the current behavior is 100% better than the previous one. It indeed needs to be better documented, though.
that would be nice.
I can't find my way through the texinfo documentation. Too much manual section management. Could we be using something nicer? ReST? Exscribe? Gary King's CL-Markdown?
It would be nice if from your file was extracted a test for the if someone would enumerate the cases which are supported. i will try again. as i understand your demurral, relative to the tested enumeration[2], one should eliminate the cases which involve string designators for logical pathnames. should anything else be removed? are that any additional cases for which support should be tested?
Non-logical pathnames are out. Namestrings without #p"..." are out. Pathnames made with (make-pathname ...) are in. /-separated pseudo-namestrings relative paths are in. Tests should include cases where the source-file-type is "lisp", NIL or :directory. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Politicians are like rats. What they steal for themselves is minuscule compared to what they destroy getting it.
On 3/8/10 Mar 8 -6:35 PM, Faré wrote: [...snip...] I have committed a modification to the info file that attempts to reflect the following. For now the discussion will appear in the "defsystem grammar" section of the manual; I don't have time for farther-reaching changes right now.
Now supported (1.630): 1- the name is a string or symbol (the name of which will be downcased). It will be interpreted by merge-component-name-type as a /-separated path, using the type specified by the component class if any.
Added discussion of the way a simple-component-name (in the terms of the grammar) are interpreted. At the expense of being a nitpicker, the obvious question is "what happens if there is no type specified by the component class? Are we guaranteeing then that the type will be empty? I'm not documenting the type behavior for now, because it belongs somewhere else...
2- the :pathname specification overrides the name. It is interpreted similarly if a symbol or a string. It can also be a pathname, in which case it is not further interpreted.
What does it mean to say that it overrides the name? What happens if we have (:file "foo/bar" :pathname "bletch") do we get "bletch/foo/bar" or "bletch/bar". If the latter, I think it would be better simply to define this as an error condition. I will wait for instructions about how to document this.
3- the parent's pathname from is an absolute pathname, with proper host and device.
?
4- the pathname obtained from 1 or 2 is merged with the one from 3 as with merge-pathnames*, i.e. if it's relative, then the parent's host and device are used.
Actually, now that we are using merge-pathnames* instead of merge-pathnames, I find that the default argument to merge-component-name-type becomes useless. Unless anyone objects, I will remove it in ASDF 1.631 (and rename the function).
Regarding 2, note that the only portable ways to obtain a pathname are a- #p"LOGICAL-HOST:logical;path;name" b- #.(make-pathname ...) c- More #. magic with merge-pathnames, *load-pathname*, *load-truename*, (user-homedir-pathname), etc.
Got it, I think.
In particular, non-logical namestrings are NOT portable, MUST NOT be used in a portable .asd file, and are otherwise not supported.
Therefore, I think the current behavior is 100% better than the previous one. It indeed needs to be better documented, though.
that would be nice.
I can't find my way through the texinfo documentation. Too much manual section management. Could we be using something nicer? ReST? Exscribe? Gary King's CL-Markdown?
Texinfo is standard and the toolchain is available to everyone. CL-Markdown has a big dependency chain, and a brittle one. If you use emacs, it will take care of all of the manual section management: Texinfo > Update every node, Texinfo > Node..., Texinfo > Create menu.... Similarly, emacs will take care of building for you, to a number of different formats. So far, I'm pretty much the only one editing the manual. I promise to stop if the format changes. Best, Robert
good evening; i have reformulated the test cases and run them through several implementations.[0] 1. i had thought (eg. [1]) that abcl and asdf were compatible. is there some special version involved? the cl.net release failed to load.[2] 2. the description of the interaction between a :pathname specification and a default file type was not clear. i added various combinations to the tests. please advise as to which ones are intended to be supported. 3. neither the syntax nor the semantics of the "lisp", nil, :directory use cases is clear. On 2010-03-09, at 01:35 , Faré wrote:
[ ... ]
Now supported (1.630): 1- the name is a string or symbol (the name of which will be downcased). It will be interpreted by merge-component-name-type as a /-separated path, using the type specified by the component class if any. 2- the :pathname specification overrides the name. It is interpreted similarly if a symbol or a string. It can also be a pathname, in which case it is not further interpreted.
if "overrides" and "not further interpreted" are intended to mean, that the :pathname specification must include any eventual file type, then the documentation should be _very_ clear about that. from my first look at the test results, it seems that this restriction applies, but i am still uncertain. (see [2] above)
3- the parent's pathname from is an absolute pathname, with proper host and device.
this is not clear. is not a parent pathname - as a component- pathname, v/s a component-relative-pathname, always necessarily absolute?
4- the pathname obtained from 1 or 2 is merged with the one from 3 as with merge-pathnames*, i.e. if it's relative, then the parent's host and device are used.
Actually, now that we are using merge-pathnames* instead of merge-pathnames, I find that the default argument to merge-component-name-type becomes useless. Unless anyone objects, I will remove it in ASDF 1.631 (and rename the function).
Regarding 2, note that the only portable ways to obtain a pathname are a- #p"LOGICAL-HOST:logical;path;name" b- #.(make-pathname ...) c- More #. magic with merge-pathnames, *load-pathname*, *load-truename*, (user-homedir-pathname), etc.
a and b were the only ways which were present in the tests. are there any 'c' forms which should be added. please endeavour to make clear in asdf documentation, that a and b are the only ways which asdf supports. as #p"..." is defined as equivalent to (parse-namestring "...") any limitation is that of asdf support, not of general portability.
In particular, non-logical namestrings are NOT portable, MUST NOT be used in a portable .asd file, and are otherwise not supported.
there were never any in the test. all namestrings were logical pathname designators. which are intended to be portable.
[ ... ]
It would be nice if from your file was extracted a test for the if someone would enumerate the cases which are supported. i will try again. as i understand your demurral, relative to the tested enumeration[2], one should eliminate the cases which involve string designators for logical pathnames. should anything else be removed? are that any additional cases for which support should be tested?
Non-logical pathnames are out.
there were never any in the test.
Namestrings without #p"..." are out.
ok. meaning physical pathname namestrings and logical pathname namestrings?
Pathnames made with (make-pathname ...) are in.
there were lots of those. are there any variations which should be added?
/-separated pseudo-namestrings relative paths are in.
please describe the syntax for pathname pseudo-namestrings. i have added pathname specifications to the tests[3] which mirror clhs 19.3.1, but a. use '/' instead of ';' as the (relative)-directory-marker; b. follow the unix convention for "relative" rather than the logical pathname convention; c. exclude hosts is that the intended syntax?
Tests should include cases where the source-file-type is "lisp", NIL or :directory.
i don't understand how these are intended to be used. please give me some examples. i can leave the server up for the next couple of hours. --- [0] : http://ec2-184-73-77-22.compute-1.amazonaws.com/test/ [1] : http://abcl-dev.blogspot.com/2010/02/require-now-works-with- asdf-systems.html [2] : http://ec2-184-73-77-22.compute-1.amazonaws.com/test/output.txt [3] : http://ec2-184-73-77-22.compute-1.amazonaws.com/test/asdf- pathname-test.lisp
On 2010-03-09, at 19:00 , james anderson wrote:
good evening;
i have reformulated the test cases and run them through several implementations.[0]
i have pushed these tot eh asdf-x repository[1] in case anyone wants to suggest changes. --- [1] : http://github.com/lisp/de.setf.asdf.x/tree/master/test/
Dear James,
i have reformulated the test cases and run them through several implementations.[0]
Thanks a lot!
1. i had thought (eg. [1]) that abcl and asdf were compatible. is there some special version involved? the cl.net release failed to load.[2]
Oh my, the asdf:around fiasco again. ABCL apparently uses an old version of ASDF to avoid a lot DEFINE-METHOD-COMBINATION. Couldn't we instead just have a do-perform function do the wrapping stuff and call perform gf, and skip that method-combination complication? Is there a good reason to use this method-combination protocol? Does anyone rely on either defining asdf:around methods or on calling perform directly and having it do the restart magic?
2. the description of the interaction between a :pathname specification and a default file type was not clear. i added various combinations to the tests. please advise as to which ones are intended to be supported.
root should be (make-pathname :name nil :type nil :version nil :defaults *load-pathname*) These are not currently portably valid: * module "./" For completeness, these should be added: * system nil (default from *load-truename*) * module "module2/submodule1/" * module "module2/submodule1" (same as above, without ending /) * for completeness, "module2/submodule4.foo/" and "module2/submodule4.foo" * source test to be split between :file lisp module and :static-file data file * :file "file2.lisp" means #p"file2.lisp.lisp" * :static-file "file2.lisp" means #p"file2.lisp" * :file "module1-1/file3.lisp" means #p"module1-1/file3.lisp.lisp" (assuming /) * :static-file "module1-1/file3.lisp" means #p"module1-1/file3.lisp" * similarly for absolute path as a string.
3. neither the syntax nor the semantics of the "lisp", nil, :directory use cases is clear.
Yes, it hasn't been documented so far. My bad. At least it now has a well-defined meaning, as opposed to the previous mess. a- when the source-file-type of a component is NIL, then the file type is read from the last /-separated component of the string as the last dot-separated component (unless there's only one dot and it's the first character, in which case the type is NIL and that's the name). b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name. c- when the source-file-type of a component is :DIRECTORY, then all /-separated components of the string including the last one are interpreted as directories.
Now supported (1.630): 1- the name is a string or symbol (the name of which will be downcased). It will be interpreted by merge-component-name-type as a /-separated path, using the type specified by the component class if any. 2- the :pathname specification overrides the name. It is interpreted similarly if a symbol or a string. It can also be a pathname, in which case it is not further interpreted.
if "overrides" and "not further interpreted" are intended to mean, that the :pathname specification must include any eventual file type, then the documentation should be _very_ clear about that. from my first look at the test results, it seems that this restriction applies, but i am still uncertain. (see [2] above)
Yes, currently, that's the case. That's how we allow the user to have fine-control over the file type when he so desires. If you want control you can have it - use pathnames and you're on your own. If you want automation, use a string and let ASDF parse it the way that makes obvious sense. But you're right, it's one more thing that needs to be documented. Sigh.
3- the parent's pathname from is an absolute pathname, with proper host and device.
this is not clear. is not a parent pathname - as a component- pathname, v/s a component-relative-pathname, always necessarily absolute?
Yes, it is always necessarily absolute, and therefore we can trust its host and device to matter. Unless the user is playing games of some sort.
Regarding 2, note that the only portable ways to obtain a pathname are a- #p"LOGICAL-HOST:logical;path;name" b- #.(make-pathname ...) c- More #. magic with merge-pathnames, *load-pathname*, *load-truename*, (user-homedir-pathname), etc.
a and b were the only ways which were present in the tests. are there any 'c' forms which should be added.
Probably no need to support c in the tests, as it's implicit from b. Just make sure some absolute paths are included in the test for b.
please endeavour to make clear in asdf documentation, that a and b are the only ways which asdf supports. as #p"..." is defined as equivalent to (parse-namestring "...") any limitation is that of asdf support, not of general portability.
Well, you can put anything in your #p"..." but because the behavior of parse-namestring is largely left unspecified, well-behaved portable system definitions should not rely on it.
In particular, non-logical namestrings are NOT portable, MUST NOT be used in a portable .asd file, and are otherwise not supported.
there were never any in the test. all namestrings were logical pathname designators. which are intended to be portable.
Good. Thanks a lot for that.
Namestrings without #p"..." are out.
ok. meaning physical pathname namestrings and logical pathname namestrings?
Yes. Strings are interpreted by ASDF::MERGE-COMPONENT-NAME-TYPE, not by CL:PARSE-NAMESTRING.
Pathnames made with (make-pathname ...) are in.
there were lots of those. are there any variations which should be added?
I think you've covered all the bases, but I'll have to take a third look.
/-separated pseudo-namestrings relative paths are in.
please describe the syntax for pathname pseudo-namestrings. i have added pathname specifications to the tests[3] which mirror clhs 19.3.1, but a. use '/' instead of ';' as the (relative)-directory-marker; b. follow the unix convention for "relative" rather than the logical pathname convention; c. exclude hosts
is that the intended syntax?
Yes. Plus the fact that either a type is provided by the component class, or none is and the type is taken from the string.
Tests should include cases where the source-file-type is "lisp", NIL or :directory.
i don't understand how these are intended to be used. please give me some examples.
See the spec in previous message. I'll have to document that. Thanks a *LOT* for your help with clarifying the semantics of pathname specifications for ASDF. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Always remember that you are unique. Just like everyone else.
On 3/9/10 Mar 9 -5:49 PM, Faré wrote:
Yes, it hasn't been documented so far. My bad. At least it now has a well-defined meaning, as opposed to the previous mess. a- when the source-file-type of a component is NIL, then the file type is read from the last /-separated component of the string as the last dot-separated component (unless there's only one dot and it's the first character, in which case the type is NIL and that's the name). b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name.
This case worries me. It seems to require that every system definer have a strongish sense of the internals of ASDF, and will give odd results when someone writes (:cl-source-file "foo.lisp") ASDF will want "foo.lisp.lisp". I know /why/ you got to this, and if you think about ASDF "from the inside" it makes sense, but "from the outside" I don't think it makes sense. Would it be so bad to change this to "when the source-file-type of a component is a string, then a component-name containing a period will cause ASDF to emit an error." This will cause the vast majority of simple cases to behave sensibly, IMO. It has the effect of making it hard to put a period in a file name. But since embedding periods in filenames in the CL pathname systems is problematic, I'd prefer making the user specifically indicate that s/he is good and bloody sure s/he wants that period to be part of the :name component by imposing some cumbersome quoting requirement. Perhaps something like "fooYESIBLOODYWELLMEANIT.bar" ;-)
c- when the source-file-type of a component is :DIRECTORY, then all /-separated components of the string including the last one are interpreted as directories.
Best, Robert
b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name.
This case worries me. It seems to require that every system definer have a strongish sense of the internals of ASDF, and will give odd results when someone writes
(:cl-source-file "foo.lisp")
ASDF will want "foo.lisp.lisp".
I know /why/ you got to this, and if you think about ASDF "from the inside" it makes sense, but "from the outside" I don't think it makes sense.
Would it be so bad to change this to
"when the source-file-type of a component is a string, then a component-name containing a period will cause ASDF to emit an error."
This will cause the vast majority of simple cases to behave sensibly, IMO. It has the effect of making it hard to put a period in a file name. But since embedding periods in filenames in the CL pathname systems is problematic, I'd prefer making the user specifically indicate that s/he is good and bloody sure s/he wants that period to be part of the :name component by imposing some cumbersome quoting requirement. Perhaps something like "fooYESIBLOODYWELLMEANIT.bar" ;-)
I'd vote against this rule you propose, because 1- I am in charge of building a large system, where some components have names such as foo-V1.1/ or bar/baz-V1.200.lisp that reflect the fact that we must deal with compatibility with various versioned protocols. I'd rather not go back to having to magically generate pathnames for them when portable names were previously possible. 2- for backwards compatibility with existing system files, the type must be optional. 3- for aesthetic reasons, I find that it's nicer if I don't have to mysteriously do "bar/baz" but "bar/baz-V1.200.lisp". I feel that the rule ".lisp is always added to the filename" is simpler and easier for newcomers to understand than the rule ".lisp is added to the filename iff there isn't a dot in the name already". [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Trying to make bits uncopyable is like trying to make water not wet. The sooner people accept this, and build business models that take this into account, the sooner people will start making money again. — Bruce Schneier
On 3/9/10 Mar 9 -9:46 PM, Faré wrote:
b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name.
This case worries me. It seems to require that every system definer have a strongish sense of the internals of ASDF, and will give odd results when someone writes
(:cl-source-file "foo.lisp")
ASDF will want "foo.lisp.lisp".
I know /why/ you got to this, and if you think about ASDF "from the inside" it makes sense, but "from the outside" I don't think it makes sense.
Would it be so bad to change this to
"when the source-file-type of a component is a string, then a component-name containing a period will cause ASDF to emit an error."
This will cause the vast majority of simple cases to behave sensibly, IMO. It has the effect of making it hard to put a period in a file name. But since embedding periods in filenames in the CL pathname systems is problematic, I'd prefer making the user specifically indicate that s/he is good and bloody sure s/he wants that period to be part of the :name component by imposing some cumbersome quoting requirement. Perhaps something like "fooYESIBLOODYWELLMEANIT.bar" ;-)
I'd vote against this rule you propose, because
1- I am in charge of building a large system, where some components have names such as foo-V1.1/ or bar/baz-V1.200.lisp that reflect the fact that we must deal with compatibility with various versioned protocols. I'd rather not go back to having to magically generate pathnames for them when portable names were previously possible.
2- for backwards compatibility with existing system files, the type must be optional.
3- for aesthetic reasons, I find that it's nicer if I don't have to mysteriously do "bar/baz" but "bar/baz-V1.200.lisp". I feel that the rule ".lisp is always added to the filename" is simpler and easier for newcomers to understand than the rule ".lisp is added to the filename iff there isn't a dot in the name already".
I think we're on the same page about that, but it seems to me that the rule in your email earlier was ".lisp" is added to the filename if there isn't a dot in the name already and if the filetype is null. I thought your proposal was that if the file was a cl-source file (and would get ".lisp" added because of source-file-type), then if you put a ".lisp" in the filename, you would get TWO copies --- ".lisp.lisp" because the .lisp in the filename would not be interpreted as a type, but as a part of the name proper. I was unhappy with that because it seemed like a complex dependency on something that's invisible to the system definer (viz. the component-file-type method). But maybe I misread your original email? Thanks, r
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Trying to make bits uncopyable is like trying to make water not wet. The sooner people accept this, and build business models that take this into account, the sooner people will start making money again. — Bruce Schneier
I think we're on the same page about that, but it seems to me that the rule in your email earlier was ".lisp" is added to the filename if there isn't a dot in the name already and if the filetype is null.
No. The current rule is: ".lisp" is added to the filename *always* if the component type is ".lisp". Which rule the user pretty much knows already, since he has plenty of components like "foo" that correspond to file "foo.lisp".
I thought your proposal was that if the file was a cl-source file (and would get ".lisp" added because of source-file-type), then if you put a ".lisp" in the filename, you would get TWO copies --- ".lisp.lisp" because the .lisp in the filename would not be interpreted as a type, but as a part of the name proper.
Yes, this is my proposal and what the current code does. On the other hand if you use #p"foo.lisp" then no additional ".lisp" is added (and neither is it added if you use #p"foo"). So if you want that behavior, you can have it, too. @james: I suppose that #p"..." names that only use portable characters could be portably accepted in pathname arguments.
I was unhappy with that because it seemed like a complex dependency on something that's invisible to the system definer (viz. the component-file-type method). But maybe I misread your original email?
The "complex" dependency already exists in legacy ASDF. Reliance upon ".lisp" being added to :file components but not :static-file components is pervasive. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] *EULA: By reading or responding to this message you agree that all my stated or unstated opinions are correct.* "EULA" — patent pending.
On 2010-03-10, at 04:46 , Faré wrote:
[ ... ]
I'd vote against this rule you propose, because
1- I am in charge of building a large system, where some components have names such as foo-V1.1/ or bar/baz-V1.200.lisp that reflect the fact that we must deal with compatibility with various versioned protocols. I'd rather not go back to having to magically generate pathnames for them when portable names were previously possible.
2- for backwards compatibility with existing system files, the type must be optional.
3- for aesthetic reasons, I find that it's nicer if I don't have to mysteriously do "bar/baz" but "bar/baz-V1.200.lisp". I feel that the rule ".lisp is always added to the filename" is simpler and easier for newcomers to understand than the rule ".lisp is added to the filename iff there isn't a dot in the name already".
checking my comprehension: this paragraph describes the way a string value for :pathname is handled. this does not happen if the argument is a pathname. ?
3- for aesthetic reasons, I find that it's nicer if I don't have to mysteriously do "bar/baz" but "bar/baz-V1.200.lisp". I feel that the rule ".lisp is always added to the filename" is simpler and easier for newcomers to understand than the rule ".lisp is added to the filename iff there isn't a dot in the name already".
checking my comprehension: this paragraph describes the way a string value for :pathname is handled. this does not happen if the argument is a pathname. ?
Indeed, when the argument is a pathname, all that happens to it is to be MERGE-PATHNAMES*'ed to the parent pathname. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] The older I grow, the more I distrust the familiar doctrine that age brings wisdom. — H.L. Mencken
On 3/9/10 Mar 9 -5:49 PM, Faré wrote:
Dear James,
i have reformulated the test cases and run them through several implementations.[0]
Thanks a lot!
1. i had thought (eg. [1]) that abcl and asdf were compatible. is there some special version involved? the cl.net release failed to load.[2]
Oh my, the asdf:around fiasco again. ABCL apparently uses an old version of ASDF to avoid a lot DEFINE-METHOD-COMBINATION.
Couldn't we instead just have a do-perform function do the wrapping stuff and call perform gf, and skip that method-combination complication?
Is there a good reason to use this method-combination protocol? Does anyone rely on either defining asdf:around methods or on calling perform directly and having it do the restart magic?
REQUEST: Let's ticket this issue in launchpad and ponder it for a while. I'm reluctant to take hasty action on this one. ISTR that decisions were taken previously, when clisp was not fully mature, that had unpleasant downstream consequences. If we're going to rip this out, I'd like it if we were sure that we were comfortable with the consequences. Also, it would be interesting to know what plans the ABCL implementers do or don't have to support D-M-C. Best, R
Faré <fahree@gmail.com> writes:
Dear James,
i have reformulated the test cases and run them through several implementations.[0]
Thanks a lot!
1. i had thought (eg. [1]) that abcl and asdf were compatible. is there some special version involved? the cl.net release failed to load.[2]
Oh my, the asdf:around fiasco again. ABCL apparently uses an old version of ASDF to avoid a lot DEFINE-METHOD-COMBINATION.
Couldn't we instead just have a do-perform function do the wrapping stuff and call perform gf, and skip that method-combination complication?
Is there a good reason to use this method-combination protocol? Does anyone rely on either defining asdf:around methods or on calling perform directly and having it do the restart magic?
A lot of people use the Stale Fasl recipe at http://www.cliki.net/asdf which was the reason that prompted introducing the custom method combination. Long form of D-M-C is in the works for ABCL from what I know. -T.
On 2010-03-10, at 00:49 , Faré wrote:
Dear James,
[ ... ]
2. the description of the interaction between a :pathname specification and a default file type was not clear. i added various combinations to the tests. please advise as to which ones are intended to be supported.
root should be (make-pathname :name nil :type nil :version nil :defaults *load-pathname*)
=> add *load-pathname* to the systems pathname argument and to create the target tree in the working directory.
These are not currently portably valid: * module "./"
=> remove these entries. elsewhere it is described that "." should work. ? add entries for "." to the module and system lists?
For completeness, these should be added: * system nil (default from *load-truename*) * module "module2/submodule1/" * module "module2/submodule1" (same as above, without ending /) * for completeness, "module2/submodule4.foo/" and "module2/ submodule4.foo"
=> ok. i understand these four, above.
* source test to be split between :file lisp module and :static- file data file * :file "file2.lisp" means #p"file2.lisp.lisp" * :static-file "file2.lisp" means #p"file2.lisp" * :file "module1-1/file3.lisp" means #p"module1-1/ file3.lisp.lisp" (assuming /) * :static-file "module1-1/file3.lisp" means #p"module1-1/file3.lisp"
? == add a :static-file as a sibling to the :file component ? == add combinations for variations in the component name
* similarly for absolute path as a string.
? do not understand
3. neither the syntax nor the semantics of the "lisp", nil, :directory use cases is clear.
Yes, it hasn't been documented so far. My bad. At least it now has a well-defined meaning, as opposed to the previous mess. a- when the source-file-type of a component is NIL, then the file type is read from the last /-separated component of the string as the last dot-separated component (unless there's only one dot and it's the first character, in which case the type is NIL and that's the name). b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name. c- when the source-file-type of a component is :DIRECTORY, then all /-separated components of the string including the last one are interpreted as directories.
? but not when it is a file or file specialization?
Now supported (1.630): 1- the name is a string or symbol (the name of which will be downcased). It will be interpreted by merge-component-name-type as a /-separated path, using the type specified by the component class if any. 2- the :pathname specification overrides the name. It is interpreted similarly if a symbol or a string. It can also be a pathname, in which case it is not further interpreted.
if "overrides" and "not further interpreted" are intended to mean, that the :pathname specification must include any eventual file type, then the documentation should be _very_ clear about that. from my first look at the test results, it seems that this restriction applies, but i am still uncertain. (see [2] above)
Yes, currently, that's the case. That's how we allow the user to have fine-control over the file type when he so desires. If you want control you can have it - use pathnames and you're on your own. If you want automation, use a string and let ASDF parse it the way that makes obvious sense. But you're right, it's one more thing that needs to be documented. Sigh.
discussed further in later messages...
[ ... ]
please describe the syntax for pathname pseudo-namestrings. i have added pathname specifications to the tests[3] which mirror clhs 19.3.1, but a. use '/' instead of ';' as the (relative)-directory-marker; b. follow the unix convention for "relative" rather than the logical pathname convention; c. exclude hosts
is that the intended syntax?
Yes. Plus the fact that either a type is provided by the component class, or none is and the type is taken from the string.
? still no clear one this. please describe a few examples with the intended effective component pathname.
On 3/10/10 Mar 10 -3:40 AM, james anderson wrote:
On 2010-03-10, at 00:49 , Faré wrote:
Dear James,
These are not currently portably valid: * module "./"
=> remove these entries. elsewhere it is described that "." should work.
Question of clarification: is "elsewhere" somewhere in the manual? If so, and if it is the feeling of the ASDF community that these should NOT work, please send me a pointer to the location, and I will excise it. As an aside, I'm not a big fan of having syntax that is read but that doesn't behave. Would it be possible to try to trap these cases and raise errors for them? Thanks, r
good afternoon; On 2010-03-10, at 14:47 , Robert Goldman wrote:
On 3/10/10 Mar 10 -3:40 AM, james anderson wrote:
On 2010-03-10, at 00:49 , Faré wrote:
Dear James,
These are not currently portably valid: * module "./"
=> remove these entries. elsewhere it is described that "." should work.
Question of clarification: is "elsewhere" somewhere in the manual?
no, i had meant, somewhere in the recent correspondence. as i cannot now find it, i must have imagined it. unless faré reiterates it, i will banish it from my thoughts.
If so, and if it is the feeling of the ASDF community that these should NOT work, please send me a pointer to the location, and I will excise it.
As an aside, I'm not a big fan of having syntax that is read but that doesn't behave.
Would it be possible to try to trap these cases and raise errors for them?
Thanks, r
These are not currently portably valid: * module "./"
=> remove these entries. elsewhere it is described that "." should work. ? add entries for "." to the module and system lists?
As rpg says, I don't know where that "somewhere" is, and I suspect it's bogus. If you want non-portable stuff, you can use #p".". I suppose I could add support for "." and maybe even ".." to the component name parsing function. If there is an outcry of demand from ASDF users, I will, but at present that seems overkill to me.
* source test to be split between :file lisp module and :static- file data file * :file "file2.lisp" means #p"file2.lisp.lisp" * :static-file "file2.lisp" means #p"file2.lisp" * :file "module1-1/file3.lisp" means #p"module1-1/ file3.lisp.lisp" (assuming /) * :static-file "module1-1/file3.lisp" means #p"module1-1/file3.lisp"
? == add a :static-file as a sibling to the :file component ? == add combinations for variations in the component name
add :static-file as a sibling, and note that various things behave differently, since :file always adds .lisp to a specified string whereas :static-file never adds anything (and neither ever substract).
* similarly for absolute path as a string. ? do not understand We probably want to check that these work (at least on Unix): "/foo/bar/baz.quux"
a- when the source-file-type of a component is NIL, then the file type is read from the last /-separated component of the string as the last dot-separated component (unless there's only one dot and it's the first character, in which case the type is NIL and that's the name). b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name. c- when the source-file-type of a component is :DIRECTORY, then all /-separated components of the string including the last one are interpreted as directories.
? but not when it is a file or file specialization?
When it's a file, the previous to last components are interpreted as directories, but the last is interpreted as a file name. When it's a directory, all components are interpreted as directories.
please describe the syntax for pathname pseudo-namestrings. i have added pathname specifications to the tests[3] which mirror clhs 19.3.1, but a. use '/' instead of ';' as the (relative)-directory-marker; b. follow the unix convention for "relative" rather than the logical pathname convention; c. exclude hosts
is that the intended syntax?
Yes. Plus the fact that either a type is provided by the component class, or none is and the type is taken from the string.
? still no clear one this. please describe a few examples with the intended effective component pathname.
(:module "a/b/c.d") => (make-pathname :directory '(:relative "a" "b" "c.d")) (:file "a/b/c.d") => (make-pathname :directory '(:relative "a" "b") :name "c.d" :type "lisp")) (:static-file "a/b/c.d") => (make-pathname :directory '(:relative "a" "b") :name "c" :type "d")) (:module "/a/b/c.d") => (make-pathname :directory '(:absolute "a" "b" "c.d")) (:file "/a/b/c.d") => (make-pathname :directory '(:absolute "a" "b") :name "c.d" :type "lisp")) (:static-file "/a/b/c.d") => (make-pathname :directory '(:absolute "a" "b") :name "c" :type "d")) [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Time and money spent in helping men to do more for themselves is far better than mere giving. — Henry Ford
On 2010-03-10, at 15:00 , Faré wrote:
These are not currently portably valid: * module "./"
=> remove these entries. elsewhere it is described that "." should work. ? add entries for "." to the module and system lists?
As rpg says, I don't know where that "somewhere" is, and I suspect it's bogus. If you want non-portable stuff, you can use #p".".
I suppose I could add support for "." and maybe even ".." to the component name parsing function. If there is an outcry of demand from ASDF users, I will, but at present that seems overkill to me.
ok. ? i don't put them in. or, i put them in and (as i need to anyway) add machinery to verify error cases?
* source test to be split between :file lisp module and :static- file data file * :file "file2.lisp" means #p"file2.lisp.lisp" * :static-file "file2.lisp" means #p"file2.lisp" * :file "module1-1/file3.lisp" means #p"module1-1/ file3.lisp.lisp" (assuming /) * :static-file "module1-1/file3.lisp" means #p"module1-1/file3.lisp"
? == add a :static-file as a sibling to the :file component ? == add combinations for variations in the component name
add :static-file as a sibling, and note that various things behave differently, since :file always adds .lisp to a specified string whereas :static-file never adds anything (and neither ever substract).
i think i finally am not unclear about this.
* similarly for absolute path as a string. ? do not understand We probably want to check that these work (at least on Unix): "/foo/bar/baz.quux"
still do not completely understand. i need a matrix \ initialization argument :name | :pathname <as string> <as string> <as string w/ separators> <as symbol> | <as string> <as pathname> component class \ file -> lisp-source-file ? static-file ? ... module system ? ? i have fabricated a mechanism to verify the results, but - while i am no longer completely unclear, i am not certain what the intended values are. especially since the two column regions are not independent and it's actually a 3-d table. plus the interaction because leaf component values are not independent of module and system values. the assertions in component-pathname-test.lisp were (and remain) just my best guesses. if one could fill out a table like this (yes, somehow n-dimensional) with a performance specification in each cell for which support is intended, i would be very happy.
a- when the source-file-type of a component is NIL, then the file type is read from the last /-separated component of the string as the last dot-separated component (unless there's only one dot and it's the first character, in which case the type is NIL and that's the name). b- when the source-file-type of a component is a string, then it will be the type, and the last /-separated component of the string provides the name. c- when the source-file-type of a component is :DIRECTORY, then all /-separated components of the string including the last one are interpreted as directories.
? but not when it is a file or file specialization?
When it's a file, the previous to last components are interpreted as directories, but the last is interpreted as a file name. When it's a directory, all components are interpreted as directories.
please describe the syntax for pathname pseudo-namestrings. i have added pathname specifications to the tests[3] which mirror clhs 19.3.1, but a. use '/' instead of ';' as the (relative)-directory-marker; b. follow the unix convention for "relative" rather than the logical pathname convention; c. exclude hosts
is that the intended syntax?
Yes. Plus the fact that either a type is provided by the component class, or none is and the type is taken from the string.
? still no clear one this. please describe a few examples with the intended effective component pathname.
(:module "a/b/c.d") => (make-pathname :directory '(:relative "a" "b" "c.d")) (:file "a/b/c.d") => (make-pathname :directory '(:relative "a" "b") :name "c.d" :type "lisp")) (:static-file "a/b/c.d") => (make-pathname :directory '(:relative "a" "b") :name "c" :type "d")) (:module "/a/b/c.d") => (make-pathname :directory '(:absolute "a" "b" "c.d")) (:file "/a/b/c.d") => (make-pathname :directory '(:absolute "a" "b") :name "c.d" :type "lisp")) (:static-file "/a/b/c.d") => (make-pathname :directory '(:absolute "a" "b") :name "c" :type "d"))
these help. could they be organized somehow like imagined above. especially to show the interaction with a :pathname argument and with parent location specifications.
On 10 March 2010 09:41, james anderson <james.anderson@setf.de> wrote:
These are not currently portably valid: * module "./"
ok. ? i don't put them in. or, i put them in and (as i need to anyway) add machinery to verify error cases?
Don't put them in, since we don't currently catch such non-portability.
* source test to be split between :file lisp module and :static- file data file * :file "file2.lisp" means #p"file2.lisp.lisp" * :static-file "file2.lisp" means #p"file2.lisp" * :file "module1-1/file3.lisp" means #p"module1-1/ file3.lisp.lisp" (assuming /) * :static-file "module1-1/file3.lisp" means #p"module1-1/file3.lisp"
? == add a :static-file as a sibling to the :file component ? == add combinations for variations in the component name
add :static-file as a sibling, and note that various things behave differently, since :file always adds .lisp to a specified string whereas :static-file never adds anything (and neither ever substract).
i think i finally am not unclear about this.
Then don't bother for now. If the thing gets integrated in the ASDF test suite I may extend it later.
* similarly for absolute path as a string. ? do not understand We probably want to check that these work (at least on Unix): "/foo/bar/baz.quux"
still do not completely understand. i need a matrix
Will do, eventually. In what SEXP format?
i have fabricated a mechanism to verify the results, but - while i am no longer completely unclear, i am not certain what the intended values are. OK.
especially since the two column regions are not independent and it's actually a 3-d table. plus the interaction because leaf component values are not independent of module and system values.
the assertions in component-pathname-test.lisp were (and remain) just my best guesses.
if one could fill out a table like this (yes, somehow n-dimensional) with a performance specification in each cell for which support is intended, i would be very happy.
Or cheat, and make all the results the same, varying the input as necessary so that each combo is fulfiiled. Alternatively, you could start extracting the current results from a run of the code, then we could double check that all results are expected. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] The problem with being a citizen of the world is that you don't get to travelling abroad much.
I darcs pull'ed from http://common-lisp.net/~crhodes/clx which compiled fine with ASDF 1.627. clx.asd says it's 0.7.2, NEWS says it's 0.7.3, and stassats says it's 0.7.4 (after using diff to confirm). In any case, I don't have any problem with (:relative) at least, not under SBCL. Do you have a backtrace of the error? Are you using logical pathnames? Maybe I should disable the output translation layer when using logical pathnames? [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Those who are afraid to take the next step will have wasted their entire previous journey. — Baron von Richthofen On 5 March 2010 13:06, james anderson <james.anderson@setf.de> wrote:
good evening;
on the occasion of pushing de.setf.graphics down the wire, when i built it for ccl/sbcl i did an obligatory pull on asdf and observe that something changed in the treatment of modules' component relative pathnames. with the effect that it was no longer possible possible to build clx. the clx-0-7-4 version as (:relative) specification in module pathnames which ran afoul of the asdf changes. i applied the same tactic as i have previously found effective for source file components and was able to build. the diff [1] is posted with the graphics sources.
on other semi-related matters, i can report, that i have now an s3 ami (linux-2.6.31+ubuntu++^3) with which i can boot an ec2 instance with all pieces in place to run the target lisp implementations.
--- [1] : http://github.com/lisp/de.setf.graphics/blob/master/readmes/ asdf.diff
_______________________________________________ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
On 2010-03-05, at 23:56 , Faré wrote:
I darcs pull'ed from http://common-lisp.net/~crhodes/clx which compiled fine with ASDF 1.627. clx.asd says it's 0.7.2, NEWS says it's 0.7.3, and stassats says it's 0.7.4 (after using diff to confirm).
In any case, I don't have any problem with (:relative) at least, not under SBCL. Do you have a backtrace of the error?
Are you using logical pathnames? Maybe I should disable the output translation layer when using logical pathnames?
the problem is sbcl's treatments of logical and physical hosts. the trace does not reveal much. it's just a compile-op on a file component. please see the message i have posted from another host with the component descriptions.
OK, I just committed ASDF 1.628, that tries to play well with logical-pathnames. I refactored component-relative-pathname, and introduced a utility MERGE-PATHNAMES* that considers that a relative pathname's host and device do not matter, and the host and device from the defaults should be used in this case. Can you give it a try? NOTE: when you're using logical pathnames, I'm bypassing output translations, with the idea that whoever uses logical pathnames probably wants to handle these himself. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Never explain. Your friends do not need it and your enemies will never believe you anyway. — Elbert Hubbard
participants (4)
-
Faré -
james anderson -
Robert Goldman -
Tobias C. Rittweiler