On 16 Oct 2020, at 13:13, Eric Timmons wrote:
"Robert Goldman" rpgoldman@sift.info writes:
In mine, I simply set the environment variables `ASDF_TEST_LISPS` and the variables that point to the lisp implementations. Then I call `make test-all-no-upgrades-no-stop`.
Just to clarify, it sounds from this that you don't actually care if the upgrade tests are run? Do you ever want to run them via CI? As it currently stands they're the slowest jobs, so finding some way to reduce the number of times they're run would be a win.
I run them rarely, and usually only close to a release. Or sometimes I run them just on SBCL, hoping that because SBCL tries to hold tight to the standard, it will be a good diagnostic tool.
But note that I typically run with an up-to-date version in my daily work, so I get a chance to see if what's shipping in ACL, SBCL, and CCL is going to cause problems with the upgrade (e.g., I'm not sure that CCL 11.1 is still upgradeable).
I think we could stick with the normal tests, and run upgrade rarely (maybe weekly at a time that's likely to see low load at cl.net?)