Windows will no longer be supported by ASDF, or ASDF, Windows, Cygwin -- oh, my!
I can set the configuration properly from an environment variable. But there are a number of tests that then reach out into the environment and try to reconfigure the source or otherwise read information. These are test-program.script test-try-refinding.script test-utilities.script None of these work properly when Lisp is running under cygwin. I'm inclined to think that this is an ASDF+Windows problem, and not a problem to be fixed in the tests. If you run a Lisp under cygwin, and you call some of these functions, they should simply error out, because the cygwin environment is going to give the lisp pathnames that it can't handle. Further, (a) we don't know how to call out to cygwin; (b) we can't call out to CMD.EXE when running under cygwin, because we have a polluted environment; and (c) UIOP cannot detect when it is running under cygwin. Under the circumstances, RUN-PROGRAM simply doesn't run reliably under cygwin. Worse, since we can't detect Cygwin, we really can't reliably run under windows at all, since we don't know when these functions will and won't work. This suggests the following: 1. We desperately need ASDF to be able to detect when it's running under cygwin, so it can at least error out when it's going to function wrong. 2. I don't know how to program make.bat. I will not be learning how to program make.bat. The test scripts that I can run are shell scripts. I have not been able to make Fare's lisp build and test scripts work and I am no longer trying to make them work at all, ever, anywhere. I don't believe they work any better under Cygwin, anyway. Ergo, if ASDF works on Windows, it will only be a matter of luck. Ergo, ASDF needs a windows tester and maintainer. I quite simply decline to perform this function. Someone who wants ASDF to work on Windows should step up. Unless someone steps up by the end of March, all further ASDF releases, starting with 3.1.7 will be officially Windows non-supporting. Windows patches and bug reports will be accepted. I will not make any attempt to fix Windows ASDF bugs myself. I will make only those Windows tests that I can make under Cygwin.
On Sun, Mar 20, 2016 at 11:23 PM, Robert Goldman <rpgoldman@sift.net> wrote:
I can set the configuration properly from an environment variable.
But there are a number of tests that then reach out into the environment and try to reconfigure the source or otherwise read information.
These are test-program.script test-try-refinding.script test-utilities.script
None of these work properly when Lisp is running under cygwin.
I'm inclined to think that this is an ASDF+Windows problem, and not a problem to be fixed in the tests. If you run a Lisp under cygwin, and you call some of these functions, they should simply error out, because the cygwin environment is going to give the lisp pathnames that it can't handle.
Further, (a) we don't know how to call out to cygwin; (b) we can't call out to CMD.EXE when running under cygwin, because we have a polluted environment; and (c) UIOP cannot detect when it is running under cygwin.
Under the circumstances, RUN-PROGRAM simply doesn't run reliably under cygwin. Worse, since we can't detect Cygwin, we really can't reliably run under windows at all, since we don't know when these functions will and won't work.
This suggests the following:
1. We desperately need ASDF to be able to detect when it's running under cygwin, so it can at least error out when it's going to function wrong.
2. I don't know how to program make.bat. I will not be learning how to program make.bat. The test scripts that I can run are shell scripts. I have not been able to make Fare's lisp build and test scripts work and I am no longer trying to make them work at all, ever, anywhere. I don't believe they work any better under Cygwin, anyway.
Ergo, if ASDF works on Windows, it will only be a matter of luck.
Ergo, ASDF needs a windows tester and maintainer. I quite simply decline to perform this function. Someone who wants ASDF to work on Windows should step up.
Unless someone steps up by the end of March, all further ASDF releases, starting with 3.1.7 will be officially Windows non-supporting. Windows patches and bug reports will be accepted. I will not make any attempt to fix Windows ASDF bugs myself. I will make only those Windows tests that I can make under Cygwin.
Then I guess SBCL holding back on ASDF upgrades is a good strategy after all. -- With best regards, Stas.
On 3/20/16 Mar 20 -7:07 PM, Stas Boukarev wrote:
Then I guess SBCL holding back on ASDF upgrades is a good strategy after all.
Actually, no. The state of affairs on Windows is no worse than before. My going back to the shell-script based testing simply REVEALS that ASDF and UIOP have never worked properly on Windows + Cygwin. Nothing about that has changed: if you run a CL implementation from inside Cygwin, it will inherit a Cygwin environment. Then RUN-PROGRAM will try to run local programs using CMD.EXE, with an environment set up for Cygwin. If you're lucky, it might work. But it's likely that the environment will have bad pathnames in it, and your use of RUN-PROGRAM will fail. Nothing there has changed. The only thing that has changed about that is that I have announced it. So, no, a simple dragging of the feet will not fix this problem. The only thing that will fix this problem will be for someone who cares about Lisp on Windows to commit some time to helping me get ASDF to work properly on Windows. Meanwhile, not updating means that you will fail to see bug fixes like the recent one that prevents ASDF from causing a stack overflow in the presence of cycles in the file system (which can be created by symbolic links, for example). I have seen more than one bug report about this from an SBCL user. Don't kid yourself that there's an easy answer. Cheers, r
On Mon, Mar 21, 2016 at 4:16 PM, Robert Goldman <rpgoldman@sift.net> wrote:
On 3/20/16 Mar 20 -7:07 PM, Stas Boukarev wrote:
Then I guess SBCL holding back on ASDF upgrades is a good strategy after all.
Actually, no.
The state of affairs on Windows is no worse than before. My going back to the shell-script based testing simply REVEALS that ASDF and UIOP have never worked properly on Windows + Cygwin. Nothing about that has changed: if you run a CL implementation from inside Cygwin, it will inherit a Cygwin environment. Then RUN-PROGRAM will try to run local programs using CMD.EXE, with an environment set up for Cygwin. If you're lucky, it might work. But it's likely that the environment will have bad pathnames in it, and your use of RUN-PROGRAM will fail.
Nothing there has changed. The only thing that has changed about that is that I have announced it.
So, no, a simple dragging of the feet will not fix this problem.
The only thing that will fix this problem will be for someone who cares about Lisp on Windows to commit some time to helping me get ASDF to work properly on Windows.
Meanwhile, not updating means that you will fail to see bug fixes like the recent one that prevents ASDF from causing a stack overflow in the presence of cycles in the file system (which can be created by symbolic links, for example). I have seen more than one bug report about this from an SBCL user.
Don't kid yourself that there's an easy answer.
I don't use cygwin on windows, does that mean I'm in the clear? -- With best regards, Stas.
On 3/21/16 Mar 21 -8:59 AM, Stas Boukarev wrote:
On Mon, Mar 21, 2016 at 4:16 PM, Robert Goldman <rpgoldman@sift.net> wrote:
On 3/20/16 Mar 20 -7:07 PM, Stas Boukarev wrote:
Then I guess SBCL holding back on ASDF upgrades is a good strategy after all.
Actually, no.
The state of affairs on Windows is no worse than before. My going back to the shell-script based testing simply REVEALS that ASDF and UIOP have never worked properly on Windows + Cygwin. Nothing about that has changed: if you run a CL implementation from inside Cygwin, it will inherit a Cygwin environment. Then RUN-PROGRAM will try to run local programs using CMD.EXE, with an environment set up for Cygwin. If you're lucky, it might work. But it's likely that the environment will have bad pathnames in it, and your use of RUN-PROGRAM will fail.
Nothing there has changed. The only thing that has changed about that is that I have announced it.
So, no, a simple dragging of the feet will not fix this problem.
The only thing that will fix this problem will be for someone who cares about Lisp on Windows to commit some time to helping me get ASDF to work properly on Windows.
Meanwhile, not updating means that you will fail to see bug fixes like the recent one that prevents ASDF from causing a stack overflow in the presence of cycles in the file system (which can be created by symbolic links, for example). I have seen more than one bug report about this from an SBCL user.
Don't kid yourself that there's an easy answer.
I don't use cygwin on windows, does that mean I'm in the clear?
I don't know. I don't have a way to run the tests without Cygwin. That's why I need help. Need someone with a minimal level of competence in Windows bat-files to figure out how to run the tests there. The current test and build suites are based on Make and bash. So they essentially don't work w/o Cygwin (maybe they would work with MinGW? I don't know). That means that Windows is essentially not tested. Again, this is not something new. Essentially, Windows has never been effectively tested. Dave Cooper and I have tried to test on Windows, but only with Cygwin. Is it possible that someone could make this work with Powershell? I don't really know much about the current status of the windows platform. My impression is that it would be pretty difficult to port to make everything work with just CMD.EXE, because it's so different and is relatively crude. R
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me. On Mon, Mar 21, 2016 at 5:32 PM, Robert Goldman <rpgoldman@sift.net> wrote:
On 3/21/16 Mar 21 -8:59 AM, Stas Boukarev wrote:
On Mon, Mar 21, 2016 at 4:16 PM, Robert Goldman <rpgoldman@sift.net> wrote:
On 3/20/16 Mar 20 -7:07 PM, Stas Boukarev wrote:
Then I guess SBCL holding back on ASDF upgrades is a good strategy after all.
Actually, no.
The state of affairs on Windows is no worse than before. My going back to the shell-script based testing simply REVEALS that ASDF and UIOP have never worked properly on Windows + Cygwin. Nothing about that has changed: if you run a CL implementation from inside Cygwin, it will inherit a Cygwin environment. Then RUN-PROGRAM will try to run local programs using CMD.EXE, with an environment set up for Cygwin. If you're lucky, it might work. But it's likely that the environment will have bad pathnames in it, and your use of RUN-PROGRAM will fail.
Nothing there has changed. The only thing that has changed about that is that I have announced it.
So, no, a simple dragging of the feet will not fix this problem.
The only thing that will fix this problem will be for someone who cares about Lisp on Windows to commit some time to helping me get ASDF to work properly on Windows.
Meanwhile, not updating means that you will fail to see bug fixes like the recent one that prevents ASDF from causing a stack overflow in the presence of cycles in the file system (which can be created by symbolic links, for example). I have seen more than one bug report about this from an SBCL user.
Don't kid yourself that there's an easy answer.
I don't use cygwin on windows, does that mean I'm in the clear?
I don't know. I don't have a way to run the tests without Cygwin. That's why I need help. Need someone with a minimal level of competence in Windows bat-files to figure out how to run the tests there.
The current test and build suites are based on Make and bash. So they essentially don't work w/o Cygwin (maybe they would work with MinGW? I don't know). That means that Windows is essentially not tested.
Again, this is not something new. Essentially, Windows has never been effectively tested. Dave Cooper and I have tried to test on Windows, but only with Cygwin.
Is it possible that someone could make this work with Powershell? I don't really know much about the current status of the windows platform. My impression is that it would be pretty difficult to port to make everything work with just CMD.EXE, because it's so different and is relatively crude.
R
-- With best regards, Stas.
On 3/21/16 Mar 21 -9:34 AM, Stas Boukarev wrote:
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me.
Any chance you could grab a copy of the git repo and run the tests on Windows? I'm relatively confident that everything works the same across lisp implementations on Windows now. The same tests fail for me everywhere. The tests failing, AFAICT, are mostly due to Cygwin environment bleeding into the environment used by the Lisp (which interacts with Windows through CMD.EXE). My guess is that these will fail in pretty much the same way under mingw, but I have never used mingw, so I can't really say. Does it have its own pathname syntax, the way cygwin does? (You can tell I don't use windows -- except for testing ASDF!).
Robert Goldman <rpgoldman@sift.net> writes:
On 3/21/16 Mar 21 -9:34 AM, Stas Boukarev wrote:
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me.
Any chance you could grab a copy of the git repo and run the tests on Windows?
I'm relatively confident that everything works the same across lisp implementations on Windows now. The same tests fail for me everywhere.
The tests failing, AFAICT, are mostly due to Cygwin environment bleeding into the environment used by the Lisp (which interacts with Windows through CMD.EXE). My guess is that these will fail in pretty much the same way under mingw, but I have never used mingw, so I can't really say. Does it have its own pathname syntax, the way cygwin does? (You can tell I don't use windows -- except for testing ASDF!). Two tests fail: test-run-program.script test-sysdef-asdf.script
test-sysdef-asdf fails because sb-ext:run-program doesn't yet support output redirection on windows. test-run-program because it can't find "echo" which turns out to be a shell script with echo "$@", cmd.exe has trouble executing that. Found some bsd echo.c, compiled and stuffed it instead of the shell script, the test now passes. So, ASDF works as much as it can be expected to work. -- With best regards, Stas.
Stas Boukarev <stassats@gmail.com> writes:
Robert Goldman <rpgoldman@sift.net> writes:
On 3/21/16 Mar 21 -9:34 AM, Stas Boukarev wrote:
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me.
Any chance you could grab a copy of the git repo and run the tests on Windows?
I'm relatively confident that everything works the same across lisp implementations on Windows now. The same tests fail for me everywhere.
The tests failing, AFAICT, are mostly due to Cygwin environment bleeding into the environment used by the Lisp (which interacts with Windows through CMD.EXE). My guess is that these will fail in pretty much the same way under mingw, but I have never used mingw, so I can't really say. Does it have its own pathname syntax, the way cygwin does? (You can tell I don't use windows -- except for testing ASDF!). Two tests fail: test-run-program.script test-sysdef-asdf.script
test-sysdef-asdf fails because sb-ext:run-program doesn't yet support output redirection on windows.
test-run-program because it can't find "echo" which turns out to be a shell script with echo "$@", cmd.exe has trouble executing that. Found some bsd echo.c, compiled and stuffed it instead of the shell script, the test now passes.
So, ASDF works as much as it can be expected to work. Actually, the sysdef-asdf test not only succumbs to missing output redirection it also does 0: (SB-EXT:RUN-PROGRAM "C:\\Windows\\System32\\cmd.exe" ("/c" "\"C:\\Windows\\System32\\cmd.exe\" /c make.bat driver _files") :INPUT T :OUTPUT T :WAIT T :ALLOW-OTHER-KEYS T :ERROR T :IF-INPUT-DOES-NOT-EXIST :ERROR :IF-OUTPUT-EXISTS :OVERWRITE :IF-ERROR-EXISTS :OVERWRITE :SEARCH T :IF-OUTPUT-DOES-NOT-EXIST :CREATE :IF-ERROR-DOES-NOT-EXIST :CREATE :WAIT T :INPUT :INTERACTIVE :OUTPUT :INTERACTIVE :ERROR-OUTPUT :INTERACTIVE :INPUT :INTERACTIVE :ERROR-OUTPUT :INTERACTIVE :IF-INPUT-DOES-NOT-EXIST :ERROR :IF-OUTPUT-EXISTS :OVERWRITE :IF-ERROR-OUTPUT-EXISTS :OVERWRITE :ELEMENT-TYPE :DEFAULT :EXTERNAL-FORMAT :UTF-8 :FORCE-SHELL T :DIRECTORY #P"c:/Users/stas/asdf/" :OUTPUT :INTERACTIVE)
Which results in '\"C:\Windows\System32\cmd.exe\"' is not recognized as an internal or external c ommand, operable program or batch file. the current asdf in SBCL doesn't do that. -- With best regards, Stas.
I have a fix for test-sysdef-asdf that I have checked in in 3.1.6.17. Briefly, the test previously used a call out with RUN-PROGRAM to find the build script's notion of what the set of files in ASDF is, and then compare that with what's in asdf.asd. I couldn't get that to run for me. I replaced the call out using RUN-PROGRAM with some lisp code that does the same task. Please LMK if that works now.
On 3/21/16 Mar 21 -10:47 AM, Stas Boukarev wrote:
Robert Goldman <rpgoldman@sift.net> writes:
On 3/21/16 Mar 21 -9:34 AM, Stas Boukarev wrote:
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me.
Any chance you could grab a copy of the git repo and run the tests on Windows?
I'm relatively confident that everything works the same across lisp implementations on Windows now. The same tests fail for me everywhere.
The tests failing, AFAICT, are mostly due to Cygwin environment bleeding into the environment used by the Lisp (which interacts with Windows through CMD.EXE). My guess is that these will fail in pretty much the same way under mingw, but I have never used mingw, so I can't really say. Does it have its own pathname syntax, the way cygwin does? (You can tell I don't use windows -- except for testing ASDF!). Two tests fail: test-run-program.script test-sysdef-asdf.script
test-sysdef-asdf fails because sb-ext:run-program doesn't yet support output redirection on windows.
test-run-program because it can't find "echo" which turns out to be a shell script with echo "$@", cmd.exe has trouble executing that. Found some bsd echo.c, compiled and stuffed it instead of the shell script, the test now passes.
So, ASDF works as much as it can be expected to work.
I've got some additional failures: the tests that involve re-initializing the ASDF search paths. Somehow they grab up cygwin paths (for me /cygdrive/f/....), which of course doesn't work. I could fix those tests probably -- I use a special environment variable to load ASDF on Windows under Cygwin -- but that would be just fixing the test and leaving ASDF broken. ASDF simply doesn't reliably work under Cygwin because of the Cygwin paths. How does mingw do this stuff? Cygwin gives me /cygdrive/f/ instead of f:/, and that's what shows up in pwd.... thanks, r
Mingw translates command-line arguments that look like a path into windows style pathnames if the DLL is present. On 12:43 Mon 21 Mar , Robert Goldman wrote:
On 3/21/16 Mar 21 -10:47 AM, Stas Boukarev wrote:
Robert Goldman <rpgoldman@sift.net> writes:
On 3/21/16 Mar 21 -9:34 AM, Stas Boukarev wrote:
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me.
Any chance you could grab a copy of the git repo and run the tests on Windows?
I'm relatively confident that everything works the same across lisp implementations on Windows now. The same tests fail for me everywhere.
The tests failing, AFAICT, are mostly due to Cygwin environment bleeding into the environment used by the Lisp (which interacts with Windows through CMD.EXE). My guess is that these will fail in pretty much the same way under mingw, but I have never used mingw, so I can't really say. Does it have its own pathname syntax, the way cygwin does? (You can tell I don't use windows -- except for testing ASDF!). Two tests fail: test-run-program.script test-sysdef-asdf.script
test-sysdef-asdf fails because sb-ext:run-program doesn't yet support output redirection on windows.
test-run-program because it can't find "echo" which turns out to be a shell script with echo "$@", cmd.exe has trouble executing that. Found some bsd echo.c, compiled and stuffed it instead of the shell script, the test now passes.
So, ASDF works as much as it can be expected to work.
I've got some additional failures: the tests that involve re-initializing the ASDF search paths. Somehow they grab up cygwin paths (for me /cygdrive/f/....), which of course doesn't work.
I could fix those tests probably -- I use a special environment variable to load ASDF on Windows under Cygwin -- but that would be just fixing the test and leaving ASDF broken. ASDF simply doesn't reliably work under Cygwin because of the Cygwin paths.
How does mingw do this stuff? Cygwin gives me /cygdrive/f/ instead of f:/, and that's what shows up in pwd....
thanks, r
On 3/21/16 Mar 21 -3:01 PM, Jason Miller wrote:
Mingw translates command-line arguments that look like a path into windows style pathnames if the DLL is present.
I'm wondering what happens if you have a Lisp running under Mingw. Might the environment variables that the Lisp sees be populated with pathnames that aren't good Windows PNs?
On 12:43 Mon 21 Mar , Robert Goldman wrote:
On 3/21/16 Mar 21 -10:47 AM, Stas Boukarev wrote:
Robert Goldman <rpgoldman@sift.net> writes:
On 3/21/16 Mar 21 -9:34 AM, Stas Boukarev wrote:
I never liked the idea of cygwin, mingw (via msys2) seems to be working pretty well for me.
Any chance you could grab a copy of the git repo and run the tests on Windows?
I'm relatively confident that everything works the same across lisp implementations on Windows now. The same tests fail for me everywhere.
The tests failing, AFAICT, are mostly due to Cygwin environment bleeding into the environment used by the Lisp (which interacts with Windows through CMD.EXE). My guess is that these will fail in pretty much the same way under mingw, but I have never used mingw, so I can't really say. Does it have its own pathname syntax, the way cygwin does? (You can tell I don't use windows -- except for testing ASDF!). Two tests fail: test-run-program.script test-sysdef-asdf.script
test-sysdef-asdf fails because sb-ext:run-program doesn't yet support output redirection on windows.
test-run-program because it can't find "echo" which turns out to be a shell script with echo "$@", cmd.exe has trouble executing that. Found some bsd echo.c, compiled and stuffed it instead of the shell script, the test now passes.
So, ASDF works as much as it can be expected to work.
I've got some additional failures: the tests that involve re-initializing the ASDF search paths. Somehow they grab up cygwin paths (for me /cygdrive/f/....), which of course doesn't work.
I could fix those tests probably -- I use a special environment variable to load ASDF on Windows under Cygwin -- but that would be just fixing the test and leaving ASDF broken. ASDF simply doesn't reliably work under Cygwin because of the Cygwin paths.
How does mingw do this stuff? Cygwin gives me /cygdrive/f/ instead of f:/, and that's what shows up in pwd....
thanks, r
Robert Goldman <rpgoldman@sift.net> writes: > On 3/21/16 Mar 21 -3:01 PM, Jason Miller wrote: >> Mingw translates command-line arguments that look like a path into >> windows style pathnames if the DLL is present. > > I'm wondering what happens if you have a Lisp running under Mingw. > Might the environment variables that the Lisp sees be populated with > pathnames that aren't good Windows PNs? Running SBCL from within msys: * *default-pathname-defaults* #P"c://" * (truename "Program Files") #P"c://Program Files/" Not sure what the double slash is about, it's not present when run from cmd.exe The msys shell itself seems to be translating things to /c/Program Files/ -- With best regards, Stas.
participants (3)
-
Jason Miller -
Robert Goldman -
Stas Boukarev