ASDF 3.1.5 test suite issues on ACL (8.2 and 10.0 GM)
Hi, all. I had an spr with Faré, but I figure it's better to have this discussion on asdf-devel. First, as Faré pointed out, ACL was releasing non-release versions of ASDF. I've rectified that, and I will be using the `release' branch from now on. Second, I can't build 3.1.5 on Windows. See below. Third, I would really love to include the ASDF tests as part of the ACL test suite. It would have prevented so many headaches. To that end, I see these problems: 1. There are reported errors, even on ACL 8.2. I can send the details, but I'm sure someone else can reproduce this. I used ALLEGRO=.../alisp make t u l=allegro in a clone of the asdf repo on the release branch. I used an 8.2 alisp, which was not in my PATH, btw. I don't have any lisps in my PATH (because I want to explicitly choose the version when I run it). 2. The symbolic link issue. The test suite doesn't run at all when the source files are available via a symbolic links. I'll be happy to get some workaround (a file loaded before the tests start so I can set *resolve-symlinks* to nil?). As for the Windows build problem: ;;; Compiling file sys:;contrib;asdf.lisp ;;; (C:\src\scm\acl82.32\src\cl\src\contrib\asdf.lisp) ; While compiling (:top-level-form "asdf.lisp" 321294): Error (from error): Invalid pathname #P"cache": Expected an absolute pathname Once I can build a release version of ASDF, I'll put it out as a patch on 8.2, 9.0 and 10.0. Once the testing issues are resolved, I will immediately add it to the test suite. Thanks. Kevin
Dear Kevin, On Fri, Sep 11, 2015 at 12:55 PM, Kevin Layer <layer@franz.com> wrote:
Hi, all. I had an spr with Faré, but I figure it's better to have this discussion on asdf-devel.
Yes, my apologies for not getting back to you. I had a very busy couple of weeks at work, and debugging ASDF on Allegro/Windows slipped out of my mind.
First, as Faré pointed out, ACL was releasing non-release versions of ASDF. I've rectified that, and I will be using the `release' branch from now on.
Thanks!
Second, I can't build 3.1.5 on Windows. See below.
I have no problem buildng or using ASDF 3.1.5 with Allegro 9.0 on Windows — except for the ASDF bug, fixed in 3.1.5.7 (that also works with Allegro 9.0), that the cache is put at the wrong place (missing the cache/ component because I fumbled the / after cache). What is (user-homedir-pathname) returning for you? Could it be that it's wrongfully returning NIL? That would explain your error behavior. Note that (user-homedir-pathname) is specified to always return an (absolute?) directory pathname object; if it doesn't that's probably a bug in Allegro. Yet, it works for me.
Third, I would really love to include the ASDF tests as part of the ACL test suite. It would have prevented so many headaches. To that end, I see these problems:
1. There are reported errors, even on ACL 8.2. I can send the details, but I'm sure someone else can reproduce this. I used
ALLEGRO=.../alisp make t u l=allegro
in a clone of the asdf repo on the release branch. I used an 8.2 alisp, which was not in my PATH, btw. I don't have any lisps in my PATH (because I want to explicitly choose the version when I run it).
For test-program.script to work, you MUST export a $PATH that contains your alisp — but that PATH need not apply outside your test script or test command line, as in ALLEGRO=.../alisp PATH=.../:$PATH make l=allegro t u
2. The symbolic link issue. The test suite doesn't run at all when the source files are available via a symbolic links. I'll be happy to get some workaround (a file loaded before the tests start so I can set *resolve-symlinks* to nil?).
It's simpler if you just copy ASDF and/or use the truename for your $PWD etc.
As for the Windows build problem:
;;; Compiling file sys:;contrib;asdf.lisp ;;; (C:\src\scm\acl82.32\src\cl\src\contrib\asdf.lisp) ; While compiling (:top-level-form "asdf.lisp" 321294): Error (from error): Invalid pathname #P"cache": Expected an absolute pathname
Once I can build a release version of ASDF, I'll put it out as a patch on 8.2, 9.0 and 10.0.
All tests pass for me with Allegro 9.0 on Windows using the ASDF 3.1.5.7 in the minimakefile branch: make l=allegro test-scripts Using 3.1.5 (more unnecessarily verbose), only test-program fails, but I forgive it: make l=allegro test-lisp [Note to Robert: that's one more reason to merge the minimakefile branch — I'm unwilling to fix run-tests.sh to do the right thing wrt to using buildi.exe on Windows. And yes, I debugged make.bat on Windows while I was at it.] Since there's a cache pathname bug on Windows in 3.1.5, I recommend shipping with 3.1.5.7 (or 3.1.6, if we manage to ship faster than you do).
Once the testing issues are resolved, I will immediately add it to the test suite.
Please trace user-homedir-pathname and/or investigate why your cache gets set wrong on Windows.
On 9/11/15 Sep 11 -8:57 PM, Faré wrote: [...snip...]
Since there's a cache pathname bug on Windows in 3.1.5, I recommend shipping with 3.1.5.7 (or 3.1.6, if we manage to ship faster than you do).
I would be happy to coordinate with you to release a 3.1.6 that is 3.1.5 with fixes for Allegro (and whatever other fixes come along for the ride). As Faré reports, you will have to test with 3.1.5.7 in order to get the fix you need for Windows. If that works for you, we could bless that as 3.1.6, and move on from there. I'll try to respond to your other points, but this entire month is wildly busy for me, between work and travel. Best, r
Since there's a cache pathname bug on Windows in 3.1.5, I recommend shipping with 3.1.5.7 (or 3.1.6, if we manage to ship faster than you do).
I would be happy to coordinate with you to release a 3.1.6 that is 3.1.5 with fixes for Allegro (and whatever other fixes come along for the ride).
As Faré reports, you will have to test with 3.1.5.7 in order to get the fix you need for Windows.
Again, I had no problem running the asdf test suite with 3.1.5.7 on Allegro / Windows using the minimakefile branch. I suspect any problem Kevin had were related to Allegro somehow having (user-homedir-pathname) incorrectly returning NIL in their test configuration. I committed in 3.1.5.8 another fix for uiop:run-program on ECL, see https://gitlab.com/embeddable-common-lisp/ecl/issues/149 and though it looks like to me I didn't break anything (and fixed it on MKCL, too), this requires re-testing on all platforms, particularly on Windows (sigh).
If that works for you, we could bless that as 3.1.6, and move on from there.
While re-testing, I noticed that concatenate-fasls was silently broken by LispWorks 7.0.0. Sigh. It would be nice if we fixed that for 3.1.6, but it looks like this might not technically be a regression. :-( In any case, assuming it passes all tests on all platforms (which I haven't tested), I recommend that Franz should deliver its Lisp with the latest development version of ASDF (currently 3.1.5.8) rather than the slightly broken release 3.1.5 (notably cache location on Windows), if we don't release 3.1.6 on time. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The common argument that crime is caused by poverty is a kind of slander on the poor. — H. L. Mencken
Faré wrote:
Since there's a cache pathname bug on Windows in 3.1.5, I recommend shipping with 3.1.5.7 (or 3.1.6, if we manage to ship faster than you do).
I would be happy to coordinate with you to release a 3.1.6 that is 3.1.5 with fixes for Allegro (and whatever other fixes come along for the ride).
As Faré reports, you will have to test with 3.1.5.7 in order to get the fix you need for Windows.
Again, I had no problem running the asdf test suite with 3.1.5.7 on Allegro / Windows using the minimakefile branch.
I suspect any problem Kevin had were related to Allegro somehow having (user-homedir-pathname) incorrectly returning NIL in their test configuration.
It's definitely not that: ;; Current reader case mode: :case-sensitive-lower cl-user(1): (user-homedir-pathname) #P"C:\\Users\\layer\\" cl-user(2): I'll test tomorrow with 3.1.5.7 and let you know what I find.
I committed in 3.1.5.8 another fix for uiop:run-program on ECL, see https://gitlab.com/embeddable-common-lisp/ecl/issues/149 and though it looks like to me I didn't break anything (and fixed it on MKCL, too), this requires re-testing on all platforms, particularly on Windows (sigh).
Just to be clear, what branch should I use for my testing? master? Something else?
If that works for you, we could bless that as 3.1.6, and move on from there.
While re-testing, I noticed that concatenate-fasls was silently broken by LispWorks 7.0.0. Sigh. It would be nice if we fixed that for 3.1.6, but it looks like this might not technically be a regression. :-(
In any case, assuming it passes all tests on all platforms (which I haven't tested), I recommend that Franz should deliver its Lisp with the latest development version of ASDF (currently 3.1.5.8) rather than the slightly broken release 3.1.5 (notably cache location on Windows), if we don't release 3.1.6 on time.
As soon as I can get a version that passes tests, I'll be happy to release it as a patch on 8.2, 9.0 and 10.0. Kevin
On Mon, Sep 14, 2015 at 3:11 AM, Kevin Layer <layer@franz.com> wrote:
I suspect any problem Kevin had were related to Allegro somehow having (user-homedir-pathname) incorrectly returning NIL in their test configuration.
It's definitely not that:
;; Current reader case mode: :case-sensitive-lower cl-user(1): (user-homedir-pathname) #P"C:\\Users\\layer\\" cl-user(2):
OK. Do you have anything else in your environment that would interfere? env | grep -i 'appdata|xdg_' Can you (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) in between loading asdf and using it (with e.g. (asdf:make "asdf")) to narrow down what in uiop/configuration.lisp is going awry for your cache configuration?
I'll test tomorrow with 3.1.5.7 and let you know what I find.
Thanks. Meanwhile, fixes for ECL, MKCL, Lispworks have pushed us to 3.1.5.10.
I committed in 3.1.5.8 another fix for uiop:run-program on ECL, see https://gitlab.com/embeddable-common-lisp/ecl/issues/149 and though it looks like to me I didn't break anything (and fixed it on MKCL, too), this requires re-testing on all platforms, particularly on Windows (sigh).
Just to be clear, what branch should I use for my testing? master? Something else?
Usually, I would have recommended release. Today, let's do master. :-( (You can also use minimakefile, though it requires installing ccl or massaging the scripts to use allegro instead)
As soon as I can get a version that passes tests, I'll be happy to release it as a patch on 8.2, 9.0 and 10.0.
Thanks. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org There are two types of people in the world: People who think there are two kinds of people, and people who don't.
On Mon, Sep 14, 2015 at 11:15 AM, Faré <fahree@gmail.com> wrote:
On Mon, Sep 14, 2015 at 3:11 AM, Kevin Layer <layer@franz.com> wrote:
I suspect any problem Kevin had were related to Allegro somehow having (user-homedir-pathname) incorrectly returning NIL in their test configuration.
It's definitely not that:
;; Current reader case mode: :case-sensitive-lower cl-user(1): (user-homedir-pathname) #P"C:\\Users\\layer\\" cl-user(2):
OK. Do you have anything else in your environment that would interfere? env | grep -i 'appdata|xdg_'
Can you (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) in between loading asdf and using it (with e.g. (asdf:make "asdf")) to narrow down what in uiop/configuration.lisp is going awry for your cache configuration?
Actually, since the user-cache is computed while asdf is loading, this won't suffice. Can you do the following: (load "build/asdf.lisp") (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) (format t "*UC*: ~S~%(CUC): ~S~%" uiop:*user-cache* (uiop/configuration::compute-user-cache)) —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The only difference between a military and a terrorist often is that the former lives off taxes taken by force from the population, whereas the latter only aspires to it.
Faré wrote:
On Mon, Sep 14, 2015 at 11:15 AM, Faré <fahree@gmail.com> wrote:
On Mon, Sep 14, 2015 at 3:11 AM, Kevin Layer <layer@franz.com> wrote:
I suspect any problem Kevin had were related to Allegro somehow having (user-homedir-pathname) incorrectly returning NIL in their test configuration.
It's definitely not that:
;; Current reader case mode: :case-sensitive-lower cl-user(1): (user-homedir-pathname) #P"C:\\Users\\layer\\" cl-user(2):
OK. Do you have anything else in your environment that would interfere? env | grep -i 'appdata|xdg_'
Can you (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) in between loading asdf and using it (with e.g. (asdf:make "asdf")) to narrow down what in uiop/configuration.lisp is going awry for your cache configuration?
Actually, since the user-cache is computed while asdf is loading, this won't suffice. Can you do the following:
(load "build/asdf.lisp") (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) (format t "*UC*: ~S~%(CUC): ~S~%" uiop:*user-cache* (uiop/configuration::compute-user-cache))
I assume this is moot, given my last email. Kevin
On Mon, Sep 14, 2015 at 11:15 AM, Faré <fahree@gmail.com> wrote:
On Mon, Sep 14, 2015 at 3:11 AM, Kevin Layer <layer@franz.com> wrote:
> I suspect any problem Kevin had were related to Allegro somehow having > (user-homedir-pathname) incorrectly returning NIL in their test > configuration.
It's definitely not that:
;; Current reader case mode: :case-sensitive-lower cl-user(1): (user-homedir-pathname) #P"C:\\Users\\layer\\" cl-user(2):
OK. Do you have anything else in your environment that would interfere? env | grep -i 'appdata|xdg_'
Can you (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) in between loading asdf and using it (with e.g. (asdf:make "asdf")) to narrow down what in uiop/configuration.lisp is going awry for your cache configuration?
Actually, since the user-cache is computed while asdf is loading, this won't suffice. Can you do the following:
(load "build/asdf.lisp") (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) (format t "*UC*: ~S~%(CUC): ~S~%" uiop:*user-cache* (uiop/configuration::compute-user-cache))
I assume this is moot, given my last email.
No, it's essential — please try it. As for your previous email, the catch was that the target called "test-scripts" in the minimakefile branch is still confusingly called "test-lisp" in the master branch (I'd fix that, but need to talk to Robert about it). —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The worst thing that can happen to a good cause is, not to be skillfully attacked, but to be ineptly defended. — F. Bastiat
Faré wrote:
On Mon, Sep 14, 2015 at 11:15 AM, Faré <fahree@gmail.com> wrote:
On Mon, Sep 14, 2015 at 3:11 AM, Kevin Layer <layer@franz.com> wrote: >>> I suspect any problem Kevin had were related to Allegro somehow having >>> (user-homedir-pathname) incorrectly returning NIL in their test >>> configuration. > > It's definitely not that: > > ;; Current reader case mode: :case-sensitive-lower > cl-user(1): (user-homedir-pathname) > #P"C:\\Users\\layer\\" > cl-user(2): > OK. Do you have anything else in your environment that would interfere? env | grep -i 'appdata|xdg_'
Can you (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) in between loading asdf and using it (with e.g. (asdf:make "asdf")) to narrow down what in uiop/configuration.lisp is going awry for your cache configuration?
Actually, since the user-cache is computed while asdf is loading, this won't suffice. Can you do the following:
(load "build/asdf.lisp") (trace uiop:xdg-cache-home uiop:xdg-data-home uiop:getenv uiop:resolve-location uiop:resolve-absolute-location uiop:resolve-relative-location) (format t "*UC*: ~S~%(CUC): ~S~%" uiop:*user-cache* (uiop/configuration::compute-user-cache))
I assume this is moot, given my last email.
No, it's essential — please try it.
This whole branch of that conversation was about an error I got while compiling 3.1.5 asdf.lisp on Windows. I no longer get that error. So, if you still want me to do the above tracing, what do I do it before?
As for your previous email, the catch was that the target called "test-scripts" in the minimakefile branch is still confusingly called "test-lisp" in the master branch (I'd fix that, but need to talk to Robert about it).
On Mon, Sep 14, 2015 at 3:38 PM, Kevin Layer <layer@franz.com> wrote:
This whole branch of that conversation was about an error I got while compiling 3.1.5 asdf.lisp on Windows. I no longer get that error.
If you don't get this error — wonderful! Please proceed to the next stage (this assumes the master branch): make l=allegro test-lisp Hopefully, all tests pass. And for extra bonus: make l=allegro test-upgrade —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Lubarsky's Law of Cybernetic Entomology: There's always one more bug.
Faré wrote:
On Mon, Sep 14, 2015 at 3:38 PM, Kevin Layer <layer@franz.com> wrote:
This whole branch of that conversation was about an error I got while compiling 3.1.5 asdf.lisp on Windows. I no longer get that error.
If you don't get this error — wonderful!
Please proceed to the next stage (this assumes the master branch):
make l=allegro test-lisp
Hopefully, all tests pass.
The problem I mentioned before still exists: @thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp ALLEGRO=/c/acl100/alisp CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf /c/acl100/buildi.exe -I /c/acl100/alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" Could not find image file /c/acl100/alisp.dxl. To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1 It should be "... -I c:/acl100/alisp.dxl" in the build script. And: @thor[git:master]$ make l=allegro test-lisp ALLEGRO=alisp CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf ./buildi.exe -I alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" ./run-tests.sh: line 115: ./buildi.exe: No such file or directory To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1 @thor[git:master]$ this assumes buildi.exe is in . when you ALLEGRO is passed, but that's clearly not correct.
And for extra bonus:
make l=allegro test-upgrade
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Lubarsky's Law of Cybernetic Entomology: There's always one more bug.
On Mon, Sep 14, 2015 at 4:44 PM, Kevin Layer <layer@franz.com> wrote:
Faré wrote:
On Mon, Sep 14, 2015 at 3:38 PM, Kevin Layer <layer@franz.com> wrote:
This whole branch of that conversation was about an error I got while compiling 3.1.5 asdf.lisp on Windows. I no longer get that error.
If you don't get this error — wonderful!
Please proceed to the next stage (this assumes the master branch):
make l=allegro test-lisp
Hopefully, all tests pass.
The problem I mentioned before still exists:
@thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp ALLEGRO=/c/acl100/alisp I believe this should be ALLEGRO=c:\acl100\alisp ? (with proper shell escaping). If using cygwin, $(cygpath ...) can help.
CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: If using a native Windows Allegro, this should probably similarly be using Windows paths, and once again cygpath can help do the translation automatically.
PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf /c/acl100/buildi.exe -I /c/acl100/alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" Could not find image file /c/acl100/alisp.dxl. To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1
It should be "... -I c:/acl100/alisp.dxl" in the build script.
I believe on Windows, you should be using Windows paths. It is also possible that the master branch is somewhat broken on Windows, because of these path issues, in which case I recommend the minimakefile branch (note that said branch requires CCL to be installed and configured, and/or that you tweak the startup scripts to use Allegro instead of CCL; I just updated README.md and toos/asdf-tools.bat with better documentation; also make test-lisp is called make test-scripts in the minimakefile branch).
@thor[git:master]$ make l=allegro test-lisp ALLEGRO=alisp CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf ./buildi.exe -I alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" ./run-tests.sh: line 115: ./buildi.exe: No such file or directory To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1 @thor[git:master]$
this assumes buildi.exe is in . when you ALLEGRO is passed, but that's clearly not correct.
I see that our test script ./test/run-tests.sh in branch master will be confused on whether paths are Unixy or Windowsy. This may be a good reason to use branch minimakefile, that shall be happy with ALLEGRO being an absolute path in Windows format (though you might have to export ALLEGRODIR, too, to make sure it knows where to find buildi.exe). My apologies for the test system being so hard to configure on Windows (or Unix, for that matter — but Windows is worse, and less tested). Patches are of course welcome. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Skill without imagination is craftsmanship. Imagination without skill is contemporary art. — Tom Stoppard
Faré wrote:
On Mon, Sep 14, 2015 at 4:44 PM, Kevin Layer <layer@franz.com> wrote:
Faré wrote:
On Mon, Sep 14, 2015 at 3:38 PM, Kevin Layer <layer@franz.com> wrote:
This whole branch of that conversation was about an error I got while compiling 3.1.5 asdf.lisp on Windows. I no longer get that error.
If you don't get this error — wonderful!
Please proceed to the next stage (this assumes the master branch):
make l=allegro test-lisp
Hopefully, all tests pass.
The problem I mentioned before still exists:
@thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp ALLEGRO=/c/acl100/alisp I believe this should be ALLEGRO=c:\acl100\alisp ? (with proper shell escaping). If using cygwin, $(cygpath ...) can help.
The test scripts are written in BASH, so that would not be correct for them. ACL uses native Windows pathnames, but BASH doesn't. In any case, I tried it, and it didn't work. I doubled the \'s, too.
CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: If using a native Windows Allegro, this should probably similarly be using Windows paths, and once again cygpath can help do the translation automatically.
It's not *my* script that needs fixing. It's the ASDF test scripts.
PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf /c/acl100/buildi.exe -I /c/acl100/alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" Could not find image file /c/acl100/alisp.dxl. To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1
It should be "... -I c:/acl100/alisp.dxl" in the build script.
I believe on Windows, you should be using Windows paths.
/ and \ are interchangable on Windows, at the C library level. The c:/acl100/alisp.dxl works fine.
It is also possible that the master branch is somewhat broken on Windows, because of these path issues, in which case I recommend the minimakefile branch (note that said branch requires CCL to be installed and configured, and/or that you tweak the startup scripts to use Allegro instead of CCL; I just updated README.md and toos/asdf-tools.bat with better documentation; also make test-lisp is called make test-scripts in the minimakefile branch).
@thor[git:master]$ make l=allegro test-lisp ALLEGRO=alisp CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf ./buildi.exe -I alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" ./run-tests.sh: line 115: ./buildi.exe: No such file or directory To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1 @thor[git:master]$
I'm going to wait for you guys to get the Windows tests working before I use them. It is just too confusing for a non-ASDF devel like me.
this assumes buildi.exe is in . when you ALLEGRO is passed, but that's clearly not correct.
I see that our test script ./test/run-tests.sh in branch master will be confused on whether paths are Unixy or Windowsy. This may be a good reason to use branch minimakefile, that shall be happy with ALLEGRO being an absolute path in Windows format (though you might have to export ALLEGRODIR, too, to make sure it knows where to find buildi.exe).
My apologies for the test system being so hard to configure on Windows (or Unix, for that matter — but Windows is worse, and less tested).
Patches are of course welcome.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Skill without imagination is craftsmanship. Imagination without skill is contemporary art. — Tom Stoppard
@thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp
ALLEGRO=/c/acl100/alisp I believe this should be ALLEGRO=c:\acl100\alisp ? (with proper shell escaping). If using cygwin, $(cygpath ...) can help.
The test scripts are written in BASH, so that would not be correct for them. ACL uses native Windows pathnames, but BASH doesn't.
In any case, I tried it, and it didn't work. I doubled the \'s, too.
In the minimakefile branch, the test scripts are written in Common Lisp, which if you use a native implementation (CCL is used by default), uses native Windows pathnames. Which is why I recommend using it, since that might simplify configuration somewhat.
I'm going to wait for you guys to get the Windows tests working before I use them. It is just too confusing for a non-ASDF devel like me.
Tests worked for me on Windows using Allegro 9.0 last weekend with 3.1.5.7 in the minimakefile branch. I have not tried Allegro 10.0 or Allegro but I believe they would work just as well. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org There is neither early nor late to fall in love. Only the right time.
Faré wrote:
@thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp
ALLEGRO=/c/acl100/alisp I believe this should be ALLEGRO=c:\acl100\alisp ? (with proper shell escaping). If using cygwin, $(cygpath ...) can help.
The test scripts are written in BASH, so that would not be correct for them. ACL uses native Windows pathnames, but BASH doesn't.
In any case, I tried it, and it didn't work. I doubled the \'s, too.
In the minimakefile branch, the test scripts are written in Common Lisp, which if you use a native implementation (CCL is used by default), uses native Windows pathnames. Which is why I recommend using it, since that might simplify configuration somewhat.
I'm going to wait for you guys to get the Windows tests working before I use them. It is just too confusing for a non-ASDF devel like me.
Tests worked for me on Windows using Allegro 9.0 last weekend with 3.1.5.7 in the minimakefile branch. I have not tried Allegro 10.0 or Allegro but I believe they would work just as well.
Perhaps you can give me the exact command you used. I couldn't find something that worked.
On Mon, Sep 14, 2015 at 6:36 PM, Kevin Layer <layer@franz.com> wrote:
Perhaps you can give me the exact command you used. I couldn't find something that worked.
Attached are the relevant parts of an environment file that I sourced into my cygwin bash shell with . windows-env.sh I tested it to work with both the master branch and the minimakefile branch of ASDF, with both Allegro 9.0 and 10.0. You obviously have to adapt the various paths to your installation. If you use the master branch: make l=allegro test-lisp If you use the minimakefile branch, I recommend you install, configure and use CCL to drive the ASDF tests; but as explained in the README.md, you can now export LISP=allegro at which point the script asdf-tools.bat will try to use Allegro instead of CCL (I just debugged that for you on Windows and Linux): make l=allegro test-scripts However, whereas using LISP=ccl as the driver correctly invokes buildi.exe, using LISP=allegro as the driver leads to alisp.exe being called, which is not what I wanted. I haven't investigated why. Note that at some point while I was tinkering with test configuration, allegro was complaining about an incompatibility between fasl format 63 vs 66 while running test-program.script, which suggested either a discrepancy between buildi and alisp, or more simply my failing to cleanly separate allegro 9 from allegro 10. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Meta is Better. Anything you can do, I can do Meta. — Patrick Logan
I think Faré's windows-env.sh supersedes and is a superset of this, but for what it's worth here is what I have in my toplevel "run" script for asdf testing (I've attached the whole script for reference - note that it handles Mac, Windows, and Linux, and makes a separate copy of the asdf/ directory for each platform). u ##### asdf="asdf-${os}/"; if [ "$os" = windows ] ; then winasdf="$(realpath $(dirname $0))/${asdf}"; export ASDF_DEVEL_SOURCE_REGISTRY="$(cygpath -w ${winasdf});$(cygpath -w ${winasdf}/uiop/);$(cygpath -w ${winasdf}/ext)//" fi ##### On Mon, Sep 14, 2015 at 11:26 PM, Faré <fahree@gmail.com> wrote:
On Mon, Sep 14, 2015 at 6:36 PM, Kevin Layer <layer@franz.com> wrote:
Perhaps you can give me the exact command you used. I couldn't find something that worked.
Attached are the relevant parts of an environment file that I sourced into my cygwin bash shell with . windows-env.sh
I tested it to work with both the master branch and the minimakefile branch of ASDF, with both Allegro 9.0 and 10.0.
You obviously have to adapt the various paths to your installation.
If you use the master branch:
make l=allegro test-lisp
If you use the minimakefile branch, I recommend you install, configure and use CCL to drive the ASDF tests; but as explained in the README.md, you can now export LISP=allegro at which point the script asdf-tools.bat will try to use Allegro instead of CCL (I just debugged that for you on Windows and Linux):
make l=allegro test-scripts
However, whereas using LISP=ccl as the driver correctly invokes buildi.exe, using LISP=allegro as the driver leads to alisp.exe being called, which is not what I wanted. I haven't investigated why.
Note that at some point while I was tinkering with test configuration, allegro was complaining about an incompatibility between fasl format 63 vs 66 while running test-program.script, which suggested either a discrepancy between buildi and alisp, or more simply my failing to cleanly separate allegro 9 from allegro 10.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Meta is Better. Anything you can do, I can do Meta. — Patrick Logan
-- My Best, Dave Cooper genworks.com, gendl.org +1 248-330-2979
However, whereas using LISP=ccl as the driver correctly invokes buildi.exe, using LISP=allegro as the driver leads to alisp.exe being called, which is not what I wanted. I haven't investigated why.
That was because defining LISP=allegro resulted in exporting ALLEGRO= which wasn't defined with LISP=ccl, and that ALLEGRO overrides the mechanism that in lisp-invocation calls buildi.exe instead of alisp.exe. If you (Kevin, or any Allegro/Windows user) considers that it's a bug worth fixing rather than a feature, please propose an API as to how lisp-invocation should detect the correct way to invoke alisp. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Ask not what the government can do for you. Ask what the government is doing to you." — David Friedman, "The Machinery of Freedom", p. 21
On 9/15/15 Sep 15 -4:31 PM, Faré wrote:
However, whereas using LISP=ccl as the driver correctly invokes buildi.exe, using LISP=allegro as the driver leads to alisp.exe being called, which is not what I wanted. I haven't investigated why.
That was because defining LISP=allegro resulted in exporting ALLEGRO= which wasn't defined with LISP=ccl, and that ALLEGRO overrides the mechanism that in lisp-invocation calls buildi.exe instead of alisp.exe.
If you (Kevin, or any Allegro/Windows user) considers that it's a bug worth fixing rather than a feature, please propose an API as to how lisp-invocation should detect the correct way to invoke alisp.
Our test scripts have created an a morass of environment variables for invoking lisp that are very difficult to understand. The cascade from the makefile through the test scripts to the lisp-invocation library makes this particularly confusing. I think it would be worth trying to overhaul this, but the first step is understanding what's going on in lisp-invocation. I'm not entirely sure if this library is required for correct ASDF functioning, or only for testing. I think many of the bundle operations, while they may not officially require lisp-invocation, *assume* it, because they assume a portable means of using the various bundles that are built with ASDF now. I regret the inclusion of the bundle operations as substantially complicating ASDF and its maintenance. OTOH, ASDF really didn't support the "hosted-on-C" flavors of CL without the bundle operations. I would like to see them disabled and untested on conventional CL implementations: I don't think the burden on ASDF maintenance is justified by value to users. Best, r
On Wed, Sep 16, 2015 at 11:46 AM, Robert Goldman <rpgoldman@sift.net> wrote:
On 9/15/15 Sep 15 -4:31 PM, Faré wrote:
However, whereas using LISP=ccl as the driver correctly invokes buildi.exe, using LISP=allegro as the driver leads to alisp.exe being called, which is not what I wanted. I haven't investigated why.
That was because defining LISP=allegro resulted in exporting ALLEGRO= which wasn't defined with LISP=ccl, and that ALLEGRO overrides the mechanism that in lisp-invocation calls buildi.exe instead of alisp.exe.
If you (Kevin, or any Allegro/Windows user) considers that it's a bug worth fixing rather than a feature, please propose an API as to how lisp-invocation should detect the correct way to invoke alisp.
Our test scripts have created an a morass of environment variables for invoking lisp that are very difficult to understand. The cascade from the makefile through the test scripts to the lisp-invocation library makes this particularly confusing.
Yes, there's a lot of confusion going on due to too many configuration variables. The situation is only made worse because many important implementations have no simple standard way of being invoked, and can't usually be trusted to have a reliable name found on the $PATH: lispworks doesn't even come with an console-only executable, allegro has this complex number of variants, ccl has an architecture-dependent name, etc. If I could rely on a same name and on being on $PATH, that'd be much easier. clisp and sbcl kind of have that on Linux, but clisp isn't actively maintained anymore.
I think it would be worth trying to overhaul this, but the first step is understanding what's going on in lisp-invocation. I'm not entirely sure if this library is required for correct ASDF functioning, or only for testing. I think many of the bundle operations, while they may not officially require lisp-invocation, *assume* it, because they assume a portable means of using the various bundles that are built with ASDF now.
ASDF requires no library whatsoever for proper functioning, or for installation. It's only the test and release script that requires libraries at all. In the master branch, you need a Unix installation or cygwin. In the minimakefile branch, you don't even need cygwin.
I regret the inclusion of the bundle operations as substantially complicating ASDF and its maintenance. OTOH, ASDF really didn't support the "hosted-on-C" flavors of CL without the bundle operations.
You're underestimating what the bundle operations brought to ASDF 3, even on "non-C-hosted" implementations. They do work and bring a lot: * {monolithic-,}compile-bundle-op allows delivery of a system {and its dependencies,} as a single fasl. That's invaluable. * {monolithic-,}concatenate-source-op, which was critical in allowing asdf itself to be split into individual source files, relies on some the bundle-operation infrastructure. * image-op and program-op couldn't have been fully portable without bundle-operations, and wouldn't have been attempted. Sure, lib-op is currently broken on non-C-hosted implementations. I can fix that.
I would like to see them disabled and untested on conventional CL implementations: I don't think the burden on ASDF maintenance is justified by value to users.
I would like to see them working, and made to do great stuff: ideally, program-op would take the output of monolithic-lib-op (and/or all the individual lib-op's), link them into the implementation's runtime and produce a fully standalone executable. We could have that for ASDF 3.2. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Wherever you go, there you are!" Dual: "Wherever you are, thence you go." Beats "I'll start exercising as soon as I'm into shape."
Hi, On 2015-09-16 20:06 CEST, Faré <fahree@gmail.com> wrote:
On Wed, Sep 16, 2015 at 11:46 AM, Robert Goldman <rpgoldman@sift.net> wrote:
[...] I regret the inclusion of the bundle operations [...]
You're underestimating what the bundle operations brought to ASDF 3, even on "non-C-hosted" implementations. They do work and bring a lot: [...] * image-op and program-op couldn't have been fully portable without bundle-operations, and wouldn't have been attempted.
I use program-op on a regular basis and it has replaced a very complex scripting machinery which I gladly got rid of. Thanks Kambiz
* image-op and program-op couldn't have been fully portable without bundle-operations, and wouldn't have been attempted.
I use program-op on a regular basis and it has replaced a very complex scripting machinery which I gladly got rid of.
+1 it eliminated an entire project for us (hu.dwim.build)! :) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is just as difficult and dangerous to try to free a people that wants to remain servile as it is to enslave a people that wants to remain free.” — Niccolò Machiavelli (1469–1527)
On 9/14/15 Sep 14 -5:36 PM, Kevin Layer wrote:
Faré wrote:
@thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp > ALLEGRO=/c/acl100/alisp I believe this should be ALLEGRO=c:\acl100\alisp ? (with proper shell escaping). If using cygwin, $(cygpath ...) can help.
The test scripts are written in BASH, so that would not be correct for them. ACL uses native Windows pathnames, but BASH doesn't.
In any case, I tried it, and it didn't work. I doubled the \'s, too.
In the minimakefile branch, the test scripts are written in Common Lisp, which if you use a native implementation (CCL is used by default), uses native Windows pathnames. Which is why I recommend using it, since that might simplify configuration somewhat.
I'm going to wait for you guys to get the Windows tests working before I use them. It is just too confusing for a non-ASDF devel like me.
Tests worked for me on Windows using Allegro 9.0 last weekend with 3.1.5.7 in the minimakefile branch. I have not tried Allegro 10.0 or Allegro but I believe they would work just as well.
Perhaps you can give me the exact command you used. I couldn't find something that worked.
I'll try this today on windows with the GM. Do you use cygwin? I am going to stick to master; I still don't understand minimakefile, and I don't have time to learn its scripting language now. I have to get to the office to get at my windows VM; I'll report back after that. Best, r
Robert Goldman wrote:
On 9/14/15 Sep 14 -5:36 PM, Kevin Layer wrote:
Faré wrote:
>> @thor[git:master]$ ALLEGRO=/c/acl100/alisp make l=allegro test-lisp >>> ALLEGRO=/c/acl100/alisp >> I believe this should be ALLEGRO=c:\acl100\alisp ? (with proper shell escaping). >> If using cygwin, $(cygpath ...) can help.
The test scripts are written in BASH, so that would not be correct for them. ACL uses native Windows pathnames, but BASH doesn't.
In any case, I tried it, and it didn't work. I doubled the \'s, too.
In the minimakefile branch, the test scripts are written in Common Lisp, which if you use a native implementation (CCL is used by default), uses native Windows pathnames. Which is why I recommend using it, since that might simplify configuration somewhat.
I'm going to wait for you guys to get the Windows tests working before I use them. It is just too confusing for a non-ASDF devel like me.
Tests worked for me on Windows using Allegro 9.0 last weekend with 3.1.5.7 in the minimakefile branch. I have not tried Allegro 10.0 or Allegro but I believe they would work just as well.
Perhaps you can give me the exact command you used. I couldn't find something that worked.
I'll try this today on windows with the GM. Do you use cygwin?
Yes. Windows without Cygwin is nearly impossible to use. Btw, I haven't had time to try any of the suggestions, because I'm been swamped. Thanks for helping with this.
I am going to stick to master; I still don't understand minimakefile, and I don't have time to learn its scripting language now.
I have to get to the office to get at my windows VM; I'll report back after that.
Best, r
I'm able to compile ASDF 3.1.5.11 (what I get from HEAD on master) on Windows with 10.0 GM. When I try and run the test suite, I get: $ export PATH=/c/acl100:$PATH $ @thor[git:master]$ ALLEGRO=/c/acl100/alisp make t u l=allegro ALLEGRO=/c/acl100/alisp CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf /c/acl100/buildi.exe -I /c/acl100/alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" Could not find image file /c/acl100/alisp.dxl. To view full results and failures, try the following command: less -p ABORTED build/results/allegro-test.text Makefile:162: recipe for target 'test-lisp' failed make: *** [test-lisp] Error 1 @thor[git:master]$ and the allegro-test.text has in it: CL_SOURCE_REGISTRY=/home/layer/asdf/:/home/layer/asdf/uiop/:/home/layer/asdf/ext//: PWD=/home/layer/asdf/test OLDPWD=/home/layer/asdf /c/acl100/buildi.exe -I /c/acl100/alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" Could not find image file /c/acl100/alisp.dxl. /c/acl100/alisp.dxl is a Cygwin path not an ACL path. Doing this /c/acl100/buildi.exe -I c:/acl100/alisp.dxl -q -batch -e "(or\`,#.(load(string\`|script-support.lisp|))#.(asdf-test::compile-asdf-script))" gave me: ASDF compiled cleanly Looks like the scripts need some work on Windows/Allegro. Kevin
I'm in favor of 3.1.6 being HEAD on master. That way, I can make patches for all platforms. Kevin
I have a lot to catch up on this conversation: I was out all day today because of the holiday. I'll catch up with the list tomorrow or Wednesday. Sorry about the delay.
participants (6)
-
Attila Lendvai -
Dave Cooper -
Faré -
Kambiz Darabi -
Kevin Layer -
Robert Goldman