12.01.2012, 17:44, "Mark Evenson" evenson.not.org@gmail.com:

On Jan 11, 2012, at 06:24 , Anton Vodonosov wrote:

...

Hello.

I am running tests of some lisp libraries on various cl implementations.

Neat.  Do you publish results?

Yes.

Results are here: http://common-lisp.net/project/cl-test-grid/

Description of the idea and current status and sources are here: https://github.com/cl-test-grid/cl-test-grid

ABCL has lot of failures in particular because many test frameworks are not ported to it.

Would there be a way for us to submit a given version of ABCL for such tests ahead of release time?

I hope so. Actually the idea that anyone can run tests. You can run tests yourself. Or create a release candidate version and some of other participants will run tests for it (as of today only two people participate in running tests - me and Robert Goldman. I hope if the project will be useful for the community more people will join).

The project is in quite early stage, but already reveals lot of failures. I hope constant running tests will help to prevent introduction of new failures while the old bugs are fixed.

Any ideas and suggestions about this project are welcome.

Best regards, - Anton

12.01.2012, 17:57, "Mark Evenson" evenson.not.org@gmail.com:

I can definitely provide results from ABCL on the monthly basis you request in the top-level README, and quite possibly more frequently if desired.

This looks like a fantastic resource for implementations!

Thanks for the positive response. I hope this project will be useful.

Running ABCL test is not difficult for me now (I anyway need to run it to understand how it works for others). I invite people to run tests if they have platforms I do not have - operation systems like NetBSD and so on. It is important for other implementations.

For ABCL is not important as it is JVM based.

So for now I wish you to fix more issued in ABCL.

You can also review the failures in the reports (I reported only one because it's a regression in the new release, therefore it should be easier to find the cause). If you will have chance to look at it and understand what it is, drop me a note please. Of course, many of these failures are not failures of ABCL, but of the libraries being tested.

One of the my closest tasks on this project is to review summarize the failures: their causes and details. When I do so I will contact you with the information I have.

Best regards, - Anton

On Jan 12, 2012, at 15:18 , Anton Vodonosov wrote:

[…]

You can also review the failures in the reports (I reported only one because it's a regression in the new release, therefore it should be easier to find the cause). If you will have chance to look at it and understand what it is, drop me a note please. Of course, many of these failures are not failures of ABCL, but of the libraries being tested.

[…]

The failure was caused by work in [r13706][] for which we will triage for [abcl-1.0.2][].

Expect it to be fixed on trunk soon.

[r13706]: http://trac.common-lisp.net/armedbear/changeset/13706 [abcl-1.0.2]: http://trac.common-lisp.net/armedbear/milestone/1.0.2