user-homedir-pathname on cmucl
Not sure how this ever worked with cmucl, but user-homedir-pathname on cmucl returns #p"home:", where "home:" is a search-list. In some cases, it looks like asdf is trying to create the cache directory and end up with a path like P"home:.cache/common-lisp/cmu-21a__21a_unicode_-macosx-x86/**/*.*.~*~" but this confuses other parts of asdf and cmucl when trying to run pathname-match-p. Perhaps the best thing is to make asdf us a custom user-homedir-pathname like what is done with other lisps. An appropriate version would be (defun user-homedir-pathname () (first (ext:search-list (cl:user-homedir-pathname)))) -- Ray
On 12/19/15 Dec 19 -2:32 PM, Raymond Toy wrote:
Not sure how this ever worked with cmucl, but user-homedir-pathname on cmucl returns #p"home:", where "home:" is a search-list. In some cases, it looks like asdf is trying to create the cache directory and end up with a path like
P"home:.cache/common-lisp/cmu-21a__21a_unicode_-macosx-x86/**/*.*.~*~"
but this confuses other parts of asdf and cmucl when trying to run pathname-match-p.
Perhaps the best thing is to make asdf us a custom user-homedir-pathname like what is done with other lisps. An appropriate version would be
(defun user-homedir-pathname () (first (ext:search-list (cl:user-homedir-pathname))))
Hi, Ray -- I will be happy to make this modification. Can you provide a test that will barf when using the current, incorrect form of user-homedir-pathname? When I run cmucl 20f, I don't have any trouble with the current ASDF, and it makes files inside ~/.cache/common-lisp/cmu-20f__20f_unicode_-macosx-x86/ just fine. So if you could give a concrete example of how this is confusing ASDF and CMUCL, I'd appreciate it. cheers, r
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 12/19/15 Dec 19 -2:32 PM, Raymond Toy wrote: >> >> Not sure how this ever worked with cmucl, but user-homedir-pathname on >> cmucl returns #p"home:", where "home:" is a search-list. In some >> cases, it looks like asdf is trying to create the cache directory and >> end up with a path like >> >> P"home:.cache/common-lisp/cmu-21a__21a_unicode_-macosx-x86/**/*.*.~*~" >> >> but this confuses other parts of asdf and cmucl when trying to run >> pathname-match-p. >> >> Perhaps the best thing is to make asdf us a custom >> user-homedir-pathname like what is done with other lisps. An >> appropriate version would be >> >> (defun user-homedir-pathname () >> (first (ext:search-list (cl:user-homedir-pathname)))) Robert> Hi, Ray -- Robert> I will be happy to make this modification. Robert> Can you provide a test that will barf when using the current, incorrect Robert> form of user-homedir-pathname? Here's a really simple test: (defun test () (pathname-match-p (cl:user-homedir-pathname) (first (ext:search-list (cl:user-homedir-pathname))))) Robert> When I run cmucl 20f, I don't have any trouble with the current ASDF, Robert> and it makes files inside Robert> ~/.cache/common-lisp/cmu-20f__20f_unicode_-macosx-x86/ just fine. Robert> So if you could give a concrete example of how this is confusing ASDF Robert> and CMUCL, I'd appreciate it. Yeah, I've been using asdf with cmucl for years and have never run into this problem; it's working just fine for me now. But I created a new local user for testing some gitlab ci integration, and it fails with a type error caused by pathname-match-p. This is what I did: 1. curl -s http://beta.quicklisp.org/archive/rt/2010-10-06/rt-20101006-git.tgz | tar xf - 2. Run cmulisp -noinit and enter (require :asdf) (load "rt-20101006-git/rt.asd") (asdf:oos 'asdf:load-op :rt :force t) Running asdf causes the error. This user is bare bones; no other lisp installed anywhere, no quicklisp, not even an .cmucl-init.lisp file. Ah, I wasn't able to reproduce this with my account, but I see now the key is that I can't have a ~/.cache directory. Removing it causes an error. -- Ray
On 12/20/15 Dec 20 -12:14 PM, Raymond Toy wrote:
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 12/19/15 Dec 19 -2:32 PM, Raymond Toy wrote: >> >> Not sure how this ever worked with cmucl, but user-homedir-pathname on >> cmucl returns #p"home:", where "home:" is a search-list. In some >> cases, it looks like asdf is trying to create the cache directory and >> end up with a path like >> >> P"home:.cache/common-lisp/cmu-21a__21a_unicode_-macosx-x86/**/*.*.~*~" >> >> but this confuses other parts of asdf and cmucl when trying to run >> pathname-match-p. >> >> Perhaps the best thing is to make asdf us a custom >> user-homedir-pathname like what is done with other lisps. An >> appropriate version would be >> >> (defun user-homedir-pathname () >> (first (ext:search-list (cl:user-homedir-pathname))))
Robert> Hi, Ray --
Robert> I will be happy to make this modification.
Robert> Can you provide a test that will barf when using the current, incorrect Robert> form of user-homedir-pathname?
Here's a really simple test:
(defun test () (pathname-match-p (cl:user-homedir-pathname) (first (ext:search-list (cl:user-homedir-pathname)))))
Hm. That doesn't seem like a good test for me to add to the suite because it fails for me even after I have applied your patch. I think what we want is this, right? (defun test () (pathname-match-p (uiop/common-lisp:user-homedir-pathname) (first (ext:search-list (cl:user-homedir-pathname))))) that fails on master, and passes with your patch. It's not particularly clear to anyone who's not familiar with CMUCL. Is there a way to illustrate the issue without reference to EXT:SEARCH-LIST? Or can you provide a comment for this? If I just drop this form into the test suite, I foresee it creating great confusion for some poor shlub (possibly me) who reads in > 6 months from now.... Looking at the context in which the error occurs (if I'm right), I think it's in APPLY-OUTPUT-TRANSLATIONS in output-translations.lisp. It seems like we're doing *SOMETHING* with USER-HOMEDIR-PATHNAME followed by merging that causes PATHNAME-MATCH-P in this function to error out. But to be honest, I have no idea what's going wrong, or why it's ASDF's fault. If you merge a pathname with USER-HOMEDIR-PATHNAME's return value, it seems to me you ought to be able to call PATHNAME-MATCH-P on the result. It seems like you're saying it's something about the way that search-lists deal with nonexistent directories. Is that right? Or is this because when we merge we aren't appropriately creating something that's a wildcard pathname? Pretty lost here, actually. Thanks, r
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 12/20/15 Dec 20 -12:14 PM, Raymond Toy wrote: >>>>>>> "Robert" == Robert Goldman >>>>>>> <rpgoldman@sift.net> writes: >> I thought I had sent a response to this, but I guess not. Robert> I think what we want is this, right? Robert> (defun test () (pathname-match-p Robert> (uiop/common-lisp:user-homedir-pathname) (first Robert> (ext:search-list (cl:user-homedir-pathname))))) Robert> that fails on master, and passes with your patch. Yeah, that's the correct version. Robert> It's not particularly clear to anyone who's not familiar Robert> with CMUCL. Is there a way to illustrate the issue Robert> without reference to EXT:SEARCH-LIST? Or can you provide The whole issue is the search-list and how pathname-match-p doesn't handle it. Here's a separate example of the issue. (setf (search-list "tmp:") '("/tmp/")) (pathname-match-p #p"/tmp/foo.lisp" "tmp:foo.lisp") gives an error. But if you had a logical-pathname host "temp", then (pathname-match-p #p"/tmp/foo.lisp" "temp:*.lisp") doesn't signal an error. It actually returns nil, which doesn't seem quite right if there's a translation ("*.lisp" "/tmp/*.lisp"). Ugh. I hate pathname issues. Robert> comment for this? If I just drop this form into the test Robert> suite, I foresee it creating great confusion for some poor Robert> shlub (possibly me) who reads in > 6 months from now.... A comment is that asdf doesn't want to use search-lists for user-homedir-pathname, Convert it to a physical pathaname. Robert> It seems like we're doing *SOMETHING* with Robert> USER-HOMEDIR-PATHNAME followed by merging that causes Robert> PATHNAME-MATCH-P in this function to error out. Robert> But to be honest, I have no idea what's going wrong, or Robert> why it's ASDF's fault. If you merge a pathname with Robert> USER-HOMEDIR-PATHNAME's return value, it seems to me you Robert> ought to be able to call PATHNAME-MATCH-P on the result. Yes, and indeed (merge-pathnames "foo.lisp" (user-homedir-pathname)) returns "home:foo.lisp" as expected. But pathname-match-p doesn't like search-lists. I guess that's cmucl's fault. Robert> It seems like you're saying it's something about the way Robert> that search-lists deal with nonexistent directories. Is Robert> that right? Robert> Or is this because when we merge we aren't appropriately Robert> creating something that's a wildcard pathname? I don't know either. I will look into making pathname-match-p understand search-lists. I'll leave it up to you on whether asdf should use a different user-homedir-pathname. I think it makes some sense for any using cmucl with a pathname-match-p fix with newer versions of asdf. -- Ray
Thanks for the response. I have a test. I'm just making sure it works on all the implementations, and then I will push it as a new ASDF version. Let me know if/when you are thinking of slurping this into the new CMUCL release, and I will make a patch release of ASDF for you. Going forward, I'd prefer to see implementations bundle an official ASDF release, instead of some random git state.... Cheers, r On 12/31/15 Dec 31 -3:54 PM, Raymond Toy wrote:
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 12/20/15 Dec 20 -12:14 PM, Raymond Toy wrote: >>>>>>> "Robert" == Robert Goldman >>>>>>> <rpgoldman@sift.net> writes: >>
I thought I had sent a response to this, but I guess not.
Robert> I think what we want is this, right?
Robert> (defun test () (pathname-match-p Robert> (uiop/common-lisp:user-homedir-pathname) (first Robert> (ext:search-list (cl:user-homedir-pathname)))))
Robert> that fails on master, and passes with your patch.
Yeah, that's the correct version.
Robert> It's not particularly clear to anyone who's not familiar Robert> with CMUCL. Is there a way to illustrate the issue Robert> without reference to EXT:SEARCH-LIST? Or can you provide
The whole issue is the search-list and how pathname-match-p doesn't handle it.
Here's a separate example of the issue.
(setf (search-list "tmp:") '("/tmp/"))
(pathname-match-p #p"/tmp/foo.lisp" "tmp:foo.lisp") gives an error.
But if you had a logical-pathname host "temp", then
(pathname-match-p #p"/tmp/foo.lisp" "temp:*.lisp") doesn't signal an error. It actually returns nil, which doesn't seem quite right if there's a translation ("*.lisp" "/tmp/*.lisp").
Ugh. I hate pathname issues.
Robert> comment for this? If I just drop this form into the test Robert> suite, I foresee it creating great confusion for some poor Robert> shlub (possibly me) who reads in > 6 months from now....
A comment is that asdf doesn't want to use search-lists for user-homedir-pathname, Convert it to a physical pathaname.
Robert> It seems like we're doing *SOMETHING* with Robert> USER-HOMEDIR-PATHNAME followed by merging that causes Robert> PATHNAME-MATCH-P in this function to error out.
Robert> But to be honest, I have no idea what's going wrong, or Robert> why it's ASDF's fault. If you merge a pathname with Robert> USER-HOMEDIR-PATHNAME's return value, it seems to me you Robert> ought to be able to call PATHNAME-MATCH-P on the result.
Yes, and indeed (merge-pathnames "foo.lisp" (user-homedir-pathname)) returns "home:foo.lisp" as expected. But pathname-match-p doesn't like search-lists. I guess that's cmucl's fault.
Robert> It seems like you're saying it's something about the way Robert> that search-lists deal with nonexistent directories. Is Robert> that right?
Robert> Or is this because when we merge we aren't appropriately Robert> creating something that's a wildcard pathname?
I don't know either.
I will look into making pathname-match-p understand search-lists. I'll leave it up to you on whether asdf should use a different user-homedir-pathname. I think it makes some sense for any using cmucl with a pathname-match-p fix with newer versions of asdf.
-- Ray
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> Thanks for the response. I have a test. I'm just making sure it works Robert> on all the implementations, and then I will push it as a new ASDF version. Cool! I've also updated cmucl's pathname-match-p to work with search-lists, so this problem should also be gone. Robert> Let me know if/when you are thinking of slurping this into the new CMUCL Robert> release, and I will make a patch release of ASDF for you. Going There will be a snapshot coming, probably this coming weekend. If you're not ready by then, no worries. We'll pick it up on the next snapshot, a month later. Robert> forward, I'd prefer to see implementations bundle an official ASDF Robert> release, instead of some random git state.... So far, cmucl has always picked up releases as they've happened. -- Ray
On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote:
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> Thanks for the response. I have a test. I'm just making sure it works Robert> on all the implementations, and then I will push it as a new ASDF version.
Cool! I've also updated cmucl's pathname-match-p to work with search-lists, so this problem should also be gone.
Robert> Let me know if/when you are thinking of slurping this into the new CMUCL Robert> release, and I will make a patch release of ASDF for you. Going
There will be a snapshot coming, probably this coming weekend. If you're not ready by then, no worries. We'll pick it up on the next snapshot, a month later.
Robert> forward, I'd prefer to see implementations bundle an official ASDF Robert> release, instead of some random git state....
So far, cmucl has always picked up releases as they've happened.
OK, I just pushed 3.1.6.8 with the fix for you. If you want to do a new snapshot, I could possibly hurry up a 3.1.7 bugfix release. Cheers, r
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote: >> So far, cmucl has always picked up releases as they've happened. Robert> OK, I just pushed 3.1.6.8 with the fix for you. Robert> If you want to do a new snapshot, I could possibly hurry up a 3.1.7 Robert> bugfix release. No need to rush. If you're ready for a 3.1.7 release very soon, then we can wait. Or we can just pick it up whenever you're ready. The next snapshot will have the pathname-match-p fix in, so it shouldn't be a problem with the currently bundled asdf 3.1.6. -- Ray
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote: Robert> OK, I just pushed 3.1.6.8 with the fix for you. Hmm. I just did a git pull --rebase, but only 3.1.6.6 is available. Am I doing something wrong? -- Ray
On 1/7/16 Jan 7 -10:44 PM, Raymond Toy wrote:
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote: Robert> OK, I just pushed 3.1.6.8 with the fix for you.
Hmm. I just did a git pull --rebase, but only 3.1.6.6 is available.
Am I doing something wrong?
I think so. I just did a git pull (not rebase), and I now have Faré's 3.1.6.9 on top of my 3.1.6.8. What's your git remote? Or is this some mysterious effect of using --rebase?
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 1/7/16 Jan 7 -10:44 PM, Raymond Toy wrote: >>>>>>> "Robert" == Robert Goldman >>>>>>> <rpgoldman@sift.net> writes: >> Robert> On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote: OK, I just Robert> pushed 3.1.6.8 with the fix for you. >> >> Hmm. I just did a git pull --rebase, but only 3.1.6.6 is >> available. >> >> Am I doing something wrong? Robert> I think so. I just did a git pull (not rebase), and I now Robert> have Faré's Robert> 3.1.6.9 on top of my 3.1.6.8. Ok, AFAICT, I do have the code (from gitlab), and I can see the commit where you bumped to version to 3.1.6.8, but there's no tag for 3.1.6.8. There's one for 3.1.6.7 and 3.1.6.9 but not .8. I'll use 3.1.6.9 for the next cmucl snapshot. Thanks. -- Ray
On 1/9/16 Jan 9 -3:13 AM, Raymond Toy wrote:
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 1/7/16 Jan 7 -10:44 PM, Raymond Toy wrote: >>>>>>> "Robert" == Robert Goldman >>>>>>> <rpgoldman@sift.net> writes: >> Robert> On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote: OK, I just Robert> pushed 3.1.6.8 with the fix for you. >> >> Hmm. I just did a git pull --rebase, but only 3.1.6.6 is >> available. >> >> Am I doing something wrong?
Robert> I think so. I just did a git pull (not rebase), and I now Robert> have Faré's Robert> 3.1.6.9 on top of my 3.1.6.8.
Ok, AFAICT, I do have the code (from gitlab), and I can see the commit where you bumped to version to 3.1.6.8, but there's no tag for 3.1.6.8. There's one for 3.1.6.7 and 3.1.6.9 but not .8.
I'll use 3.1.6.9 for the next cmucl snapshot.
Sorry; forgot the "git push origin 3.1.6.8" that's needed as part of bumping the revision. So the code went, but not the tag. It seems wrong to have random tags as the versions used in implementations. Wonder if I should run the full set of tests to bless this as 3.1.7. Is there enough time for that? There have been a reasonable number of bug fixes. Best, r
"Robert" == Robert Goldman <rpgoldman@sift.net> writes:
Robert> On 1/9/16 Jan 9 -3:13 AM, Raymond Toy wrote: >>>>>>> "Robert" == Robert Goldman >>>>>>> <rpgoldman@sift.net> writes: >> Robert> On 1/7/16 Jan 7 -10:44 PM, Raymond Toy wrote: >> >>>>>>> "Robert" == Robert Goldman >> >>>>>>> <rpgoldman@sift.net> writes: >> >> Robert> On 1/4/16 Jan 4 -11:48 AM, Raymond Toy wrote: OK, I just Robert> pushed 3.1.6.8 with the fix for you. >> >> >> >> Hmm. I just did a git pull --rebase, but only 3.1.6.6 is >> >> available. >> >> >> >> Am I doing something wrong? >> Robert> I think so. I just did a git pull (not rebase), and I now Robert> have Faré's Robert> 3.1.6.9 on top of my 3.1.6.8. >> >> Ok, AFAICT, I do have the code (from gitlab), and I can see the >> commit where you bumped to version to 3.1.6.8, but there's no >> tag for 3.1.6.8. There's one for 3.1.6.7 and 3.1.6.9 but not >> .8. >> >> I'll use 3.1.6.9 for the next cmucl snapshot. >> Robert> Sorry; forgot the Robert> "git push origin 3.1.6.8" that's needed as part of bumping Robert> the revision. Robert> So the code went, but not the tag. Yeah, I saw the commit logs. Robert> It seems wrong to have random tags as the versions used in Robert> implementations. Wonder if I should run the full set of Robert> tests to bless this as 3.1.7. Is there enough time for Robert> that? There have been a reasonable number of bug fixes. Too late. :-( I tagged the snapshot already. Don't worry. We try to make sure the snapshots work, but don't guarantee it; they have been broken a few times before. Whenever you make a 3.1.7, I'll be sure to pick it up for the snapshot. -- Ray
participants (2)
-
Raymond Toy -
Robert Goldman