Dear Anton,
When we tested with cl-test-grid, ABCL, CMUCL, CCL, CLISP, ECL, SBCL were tested on 2.31.8 http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-21.html
The latest cl-test-grid cycle was for ASDF 2.32.35, it was tested on CCL, CLISP, ECL, SBCL on Linux. http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-23.html
So, as you see we ended up testing last fixes only on some lisps, not all lisps. Probably that's because it was stabilizing period and the fixes were for concrete bugs.
Still, we didn't make a firm "rollback" point - an ASDF version where we ensured we have no regressions on all lisps. For now I suggest to consider 2.32.35 as more or less reliable previous comparison point, where all regressions were proved to be not ASDF bugs, but bugs in libraries, and were reported to the library maintainers.
I hope in this ASDF release, we will test the released version on maximum number of lisps, so that in future we will have reliable comparison point.
Yes. This time, let's use 2.32.35 as our baseline, and if you can, test both ASDF 3.0.3 as the current stable, and 3.1.0.32 (current HEAD) as our release candidate for 3.1.1.
I don't actually expect any change in cl-test-grid results between all these versions, except maybe that my own software (e.g. lisp-interface-library) might possibly fail on older versions of ASDF.
But then, you'd have to check with both old and new Quicklisp.
I am going to run tests of both previous stable ASDF version and the current HEAD on the same Quicklisp - 2013-12-13.
Thanks a lot!
I suppose that you should compare with whatever comes with the implementation.
OK, we can do this too - we will compare result for the latest ASDF with the results for the "main", unpatched quicklisp, which uses ASDF coming with the implementation.
It also would be nice to compare with some concrete previous stable ASDF version, as described above, to be sure we cover full ASDF evolution on all lisps.
Don't overdo it. I would do it with maybe 2.26 (default from Quicklisp), 2.32.35 (previous tested), 3.0.3 (latest stable) and 3.1.0.32 (release candidate), and that's already a lot.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org It may be bad manners to talk with your mouth full, but it isn't too good either if you speak when your head is empty.
31.12.2013, 12:32, "Faré" fahree@gmail.com:
Don't overdo it. I would do it with maybe 2.26 (default from Quicklisp), 2.32.35 (previous tested), 3.0.3 (latest stable) and 3.1.0.32 (release candidate), and that's already a lot.
OK.
HEAD has 3 commits after 3.1.0.32. The tests are started for HEAD. Should we drop it and run for 3.1.0.32 or HEAD is better?
On Tue, Dec 31, 2013 at 3:36 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
31.12.2013, 12:32, "Faré" fahree@gmail.com:
Don't overdo it. I would do it with maybe 2.26 (default from Quicklisp), 2.32.35 (previous tested), 3.0.3 (latest stable) and 3.1.0.32 (release candidate), and that's already a lot.
OK.
HEAD has 3 commits after 3.1.0.32. The tests are started for HEAD. Should we drop it and run for 3.1.0.32 or HEAD is better?
I recommend using the latest HEAD. The code should be the same (or I would have bumped the version). Changes only happened in documentation, build and test files.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I am an atheist, thank God!
SBCL and CCL results have arrived.
Diff between ASDF HEAD and unpatched quicklisp: http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-24.html
Testing of ASDF HEAD is done by copying new asdf.lisp to to quicklisp/asdf.lisp and changing (defvar *required-asdf-version* "3.1.0.32") in quicklisp/setup.lisp.
This means quicklis first does (reqire 'asdf) and load the ASDF version provided by lisp implementation, and then upgrates to the newer ASDF.
31.12.2013, 12:42, "Faré" fahree@gmail.com:
On Tue, Dec 31, 2013 at 3:36 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
31.12.2013, 12:32, "Faré" fahree@gmail.com:
Don't overdo it. I would do it with maybe 2.26 (default from Quicklisp), 2.32.35 (previous tested), 3.0.3 (latest stable) and 3.1.0.32 (release candidate), and that's already a lot.
OK.
HEAD has 3 commits after 3.1.0.32. The tests are started for HEAD. Should we drop it and run for 3.1.0.32 or HEAD is better?
I recommend using the latest HEAD. The code should be the same (or I would have bumped the version). Changes only happened in documentation, build and test files.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I am an atheist, thank God!
When testing on CMUCL (cmu-snapshot-2013-04__20d_unicode_-linux-x86) every system fails with this error:
Error in function KERNEL:CLASS-TYPEP: Class is currently invalid: #<KERNEL::STANDARD-CLASS ASDF/SYSTEM:SYSTEM {585CFA1D}> [Condition of type SIMPLE-ERROR]
Looks like an ASDF upgrade problem.
Yes, ASDF upgrade is completely broken on CMUCL, due to undetermined PCL issues. I've tried punting harder on CMUCL (by having a #-cmu in headers.lisp before (< existing-version 2.27), but it still breaks the upgrade test, in other ways.
It would be really nice if a CMUCL hacker and/or PCL expert would fix CMUCL to not break the upgrade.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org There are a thousand hacking at the branches of evil to one who is striking at the root. — Thoreau
On Tue, Dec 31, 2013 at 8:33 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
When testing on CMUCL (cmu-snapshot-2013-04__20d_unicode_-linux-x86) every system fails with this error:
Error in function KERNEL:CLASS-TYPEP: Class is currently invalid: #<KERNEL::STANDARD-CLASS ASDF/SYSTEM:SYSTEM {585CFA1D}> [Condition of type SIMPLE-ERROR]
Looks like an ASDF upgrade problem.
The failures in http://cl-test-grid.appspot.com/blob?key=1v5pgltcap are interesting.
The CCL specific failures, I fear I can't explain, and won't even try.
The exscribe failure has no ASDF in its backtrace; this suggests that it's a Quicklisp failure, probably due to it being unable to parse (:feature :exscribe-typeset :cl-typesetting) as a dependency, and recursing into a non-existent system exscribe-typeset, when it's actually a feature that decides whether or not to depend on cl-typesetting. Admittedly, I should probably have designed things differently, and indeed have had a system exscribe-typeset that unconditionally depends on both and hooks into exscribe. What bothers me, then, is that the test only fails with the newer ASDF; why doesn't it fail with the old ASDF? What effective version of ASDF is loaded? Could cl-test-grid print that information in its header?
Regarding the lil failure: is quicklisp somehow removing ASDF::SYSDEF-PACKAGE-SYSTEM-SEARCH from ASDF::*SYSTEM-DEFINITION-SEARCH-FUNCTIONS* ? What is the value of the latter variable? But then why is it working with the old ASDF? My guess is that's because it does have ASDF/PACKAGE-SYSTEM builtin, so it loads it from the add-on system ASDF-PACKAGE-SYSTEM that installs the search function dropped by quicklisp. However, ~/quicklisp/quicklisp/setup.lisp looks like it's doing the right thing wrt *SYSTEM-DEFINITION-SEARCH-FUNCTIONS*, and I don't see cl-test-grid touching it, so I'm not so sure anymore, and I'm a bit baffled actually.
Note that at home, I have no trouble loading either lil or exscribe, with or without quicklisp, with or without the new asdf, using SBCL 1.1.13.41.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Every technique is first developed, then used, important, obsolete, normalized, and finally understood.
On Tue, Dec 31, 2013 at 6:40 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
SBCL and CCL results have arrived.
Diff between ASDF HEAD and unpatched quicklisp: http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-24.html
Testing of ASDF HEAD is done by copying new asdf.lisp to to quicklisp/asdf.lisp and changing (defvar *required-asdf-version* "3.1.0.32") in quicklisp/setup.lisp.
This means quicklis first does (reqire 'asdf) and load the ASDF version provided by lisp implementation, and then upgrates to the newer ASDF.
31.12.2013, 12:42, "Faré" fahree@gmail.com:
On Tue, Dec 31, 2013 at 3:36 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
31.12.2013, 12:32, "Faré" fahree@gmail.com:
Don't overdo it. I would do it with maybe 2.26 (default from Quicklisp), 2.32.35 (previous tested), 3.0.3 (latest stable) and 3.1.0.32 (release candidate), and that's already a lot.
OK.
HEAD has 3 commits after 3.1.0.32. The tests are started for HEAD. Should we drop it and run for 3.1.0.32 or HEAD is better?
I recommend using the latest HEAD. The code should be the same (or I would have bumped the version). Changes only happened in documentation, build and test files.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I am an atheist, thank God!
01.01.2014, 02:40, "Faré" fahree@gmail.com:
The failures in http://cl-test-grid.appspot.com/blob?key=1v5pgltcap are interesting.
The CCL specific failures, I fear I can't explain, and won't even try.
The exscribe failure has no ASDF in its backtrace; this suggests that it's a Quicklisp failure, probably due to it being unable to parse (:feature :exscribe-typeset :cl-typesetting) as a dependency, and recursing into a non-existent system exscribe-typeset, when it's actually a feature that decides whether or not to depend on cl-typesetting. What bothers me, then, is that the test only fails with the newer ASDF; why doesn't it fail with the old ASDF? What effective version of ASDF is loaded? Could cl-test-grid print that information in its header?
It turns out to be a Quicklisp bug. Indeed, quicklisp 2013-12-13 records exscribe-typeset as a dependency of exscribe.
Why it behaves differently.
To load a system, Quicklisp first tries to (asdf:find-system <system>) and only if that fails because the system is not yet downloaded, Quicklisp looks into its own records for that system and its dependencies.
If exscribe tarball was downloaded and unpacked already by previous quicklisp version, ASDF finds the system and it loads OK. But if excribe is absent on the hard drive, than Quicklisp looks into its records, sees that exscribe-typeset is a dependency, tries to find information about exscribe-typeset and fails, signalling SYSTEM-NOT-FOUND.
Admittedly, I should probably have designed things differently, and indeed have had a system exscribe-typeset that unconditionally depends on both and hooks into exscribe.
Yes, I prefer that straightforward way too.
Regarding the lil failure: is quicklisp somehow removing ASDF::SYSDEF-PACKAGE-SYSTEM-SEARCH from ASDF::*SYSTEM-DEFINITION-SEARCH-FUNCTIONS* ? What is the value of the latter variable? But then why is it working with the old ASDF? My guess is that's because it does have ASDF/PACKAGE-SYSTEM builtin, so it loads it from the add-on system ASDF-PACKAGE-SYSTEM that installs the search function dropped by quicklisp. However, ~/quicklisp/quicklisp/setup.lisp looks like it's doing the right thing wrt *SYSTEM-DEFINITION-SEARCH-FUNCTIONS*, and I don't see cl-test-grid touching it, so I'm not so sure anymore, and I'm a bit baffled actually.
I haven't reviewed the lil failures. Maybe they are caused by the same bug as exscribe.
Note that at home, I have no trouble loading either lil or exscribe, with or without quicklisp, with or without the new asdf, using SBCL 1.1.13.41.
I have added your ssh key to user testgrid at cl-test-grid.cloud.efficito.com - the machine where testing was performed. If you will need to experiment and reproduce problems, you should be able to login.
The report at http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-24.html is updated with ECL results. Also, as you can see the excribe failure is annotated by "Quicklisp bug". It's a new feature of cl-test-grid - I can record notes near the failures we clarified already, so it's easier to see what remains. Started tests for ABCL.
Best regards, - Anton
It turns out to be a Quicklisp bug. Indeed, quicklisp 2013-12-13 records exscribe-typeset as a dependency of exscribe.
OK, I modified exscribe to not use the :feature feature, but quicklisp probably needs to be fixed to support it, too. The feature was already present in late ASDF 1, but nobody used it, because it was broken until late ASDF 2. So that's QL bug #1, that triggers bug #2.
Why it behaves differently. [...]
Thanks for this brilliant analysis. Bug #2 is QL thinks a dependency exists (erroneously above, correctly below), but no .asd file directly provides it, so it fails to locate it, and dies. Yet, it obviously knows how to handle magically required asdf systems such as :sb-posix. Thus, it can probably be told how to handle systems that don't have a direct QL-managed, user-defined .asd file.
I haven't reviewed the lil failures. Maybe they are caused by the same bug as exscribe.
I think it's the same bug indeed.
I have added your ssh key to user testgrid at cl-test-grid.cloud.efficito.com
- the machine where testing was performed. If you will need to experiment
and reproduce problems, you should be able to login.
Thanks!
Well, SBCL still can't load lil, because it first tries to locate lil/interface/all and fails. I suppose it somehow works for me because I forced installation of all QL packages including lil, so that once installed, QL finds it. But QL won't do the implicit installation, because it tries to recurse on a dependency, and fails to find it. The place where to find it is under the hierarchy that contains lil.asd, courtesy of asdf/package-system.
The report at http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-24.html is updated with ECL results. Also, as you can see the excribe failure is annotated by "Quicklisp bug". It's a new feature of cl-test-grid - I can record notes near the failures we clarified already, so it's easier to see what remains. Started tests for ABCL.
Cool.
Thanks a lot for your diligent testing!
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Do not handicap your children by making their lives easy. — Robert Heinlein
The report at http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-24.html is updated with ABCL results - nothing new in ABCL.
Also, I investigated a little remaining failures, and added notes to the report.
A CCL-specific failure only happens with cl-l10n system, and only when ASDF upgrade is performed. Without ASDF upgrate the cl-l10n loads OK, while after ASDF upgrade CCL:SLOT-VALUE-USING-CLASS function fails with NO-APPLICABLE-METHOD.
If you want to reproduce, here are command lines:
# with ASDF upgrade rlwrap ./lisps/ccl-1.9/lx86cl --no-init --load ~/quicklisp-patched/setup.lisp
# without ASDF upgrade rlwrap ./lisps/ccl-1.9/lx86cl --no-init --load ~/asdf/build/asdf.lisp --load ~/quicklisp-patched/setup.lisp
Then try (ql:quickload :cl-l10n)
And another type of errors we have is failure to load hu.dwim.computed-class+hu.dwim.logger, hu.dwim.computed-class.test and caveman-test.
It is caused by Quicklisp problem to handle :defsystem-depends-on. As can be seen from the stacktrace at http://cl-test-grid.appspot.com/blob?key=52ginnerq3 Quicklisp tires (asdf:find-system "hu.dwim.computed-class+hu.dwim.logger" nil) ;; nil is passed to error-p parameter
ASDF finds the hu.dwim.computed-class+hu.dwim.logger.asd file and while parsing it then tries to resolve :defsystem-depends-on (:hu.dwim.logger), but hu.dwim.logger is not downloaded to the local disk yet, so ASDF is unable to find :hu.dwim.logger and signals an error.
This error can happen with previous ASDF as well, we only need to have hu.dwim.computed-class already downloaded and hu.dwim.logger not downloaded.
To summarize, the testing so far reveals only two ASDF-specific issues: - ASDF upgrade on CMUCL always fails - ASDF upgrade on CCL breaks cl-l10n somehow
Best regards, - Anton
02.01.2014, 23:02, "Faré" fahree@gmail.com:
It turns out to be a Quicklisp bug. Indeed, quicklisp 2013-12-13 records exscribe-typeset as a dependency of exscribe.
OK, I modified exscribe to not use the :feature feature, but quicklisp probably needs to be fixed to support it, too. The feature was already present in late ASDF 1, but nobody used it, because it was broken until late ASDF 2. So that's QL bug #1, that triggers bug #2.
Why it behaves differently. [...]
Thanks for this brilliant analysis. Bug #2 is QL thinks a dependency exists (erroneously above, correctly below), but no .asd file directly provides it, so it fails to locate it, and dies. Yet, it obviously knows how to handle magically required asdf systems such as :sb-posix. Thus, it can probably be told how to handle systems that don't have a direct QL-managed, user-defined .asd file.
I haven't reviewed the lil failures. Maybe they are caused by the same bug as exscribe.
I think it's the same bug indeed.
I have added your ssh key to user testgrid at cl-test-grid.cloud.efficito.com - the machine where testing was performed. If you will need to experiment and reproduce problems, you should be able to login.
Thanks!
Well, SBCL still can't load lil, because it first tries to locate lil/interface/all and fails. I suppose it somehow works for me because I forced installation of all QL packages including lil, so that once installed, QL finds it. But QL won't do the implicit installation, because it tries to recurse on a dependency, and fails to find it. The place where to find it is under the hierarchy that contains lil.asd, courtesy of asdf/package-system.
The report at http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-24.html is updated with ECL results. Also, as you can see the excribe failure is annotated by "Quicklisp bug". It's a new feature of cl-test-grid - I can record notes near the failures we clarified already, so it's easier to see what remains. Started tests for ABCL.
Cool.
Thanks a lot for your diligent testing!
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Do not handicap your children by making their lives easy. — Robert Heinlein
That's very weird. The cl-l10n error is in a class unrelated to ASDF. Maybe on CCL the upgrade corrupts the CLOS data structures in a subtle way that somehow only shows up for cl-l10n, but none of the hundreds other systems?
Even weirder: if I try loading cl-l10n *after* I load the :hu.dwim.computed-class+hu.dwim.logger system (but not, say, :fare-utils), it works, though if I try to load it before, it fails. WTF? Maybe CCL fails to finalize-inheritance or some such thing, but something happens in the meantime that causes it to do it?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Economics is not about goods and services; it is about human choice and action. — Ludwig von Mises
06.01.2014, 20:05, "Faré" fahree@gmail.com:
That's very weird. The cl-l10n error is in a class unrelated to ASDF. Maybe on CCL the upgrade corrupts the CLOS data structures in a subtle way that somehow only shows up for cl-l10n, but none of the hundreds other systems?
Even weirder: if I try loading cl-l10n *after* I load the :hu.dwim.computed-class+hu.dwim.logger system (but not, say, :fare-utils), it works, though if I try to load it before, it fails. WTF? Maybe CCL fails to finalize-inheritance or some such thing, but something happens in the meantime that causes it to do it?
Yes, it looks like a CCL bug. If I find time, I'll report to openmcl-devel
I have re-run test for the latest ASDF version - 3.1.0.46 (git commit 28a5c93).
In short, this ASDF has no regressions comparing to our previous base point.
The only known issues detected are the upgrade problem on old CMUCL and the strange CCL bug affecting cl-l10n.
The reports:
with ASDF upgrade http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-25.html
without ASDF upgrade http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-26.html
The failures in the reports are the same we discussed recently and are not ASDF regressions.
Best regards, - Anton
Even weirder: if I try loading cl-l10n *after* I load the :hu.dwim.computed-class+hu.dwim.logger system (but not, say,
FTR, computed-class is a non-trivial user of CLOS.
-
Anton Vodonosov -
Attila Lendvai -
Faré