"Mark" == Mark Evenson evenson@panix.com writes:
[snip]
Mark> 1) ASDF
Mark> By the nature of how ANSI-TEST loads and runs itself, encapsulation as a Mark> "true" ASDF system (i.e. one where all the source is declared) seems a Mark> little farfetched. And maybe it is useful to be able to run the suite Mark> in places where ASDF is not available, such as when bootstrapping a new Mark> implementation. Nevertheless, we put some work into encapsulating the Mark> ANSI-SUITE into ASDF to the point that it can be loaded and the tests Mark> run that might be of interest.
This is interesting. It sounds nicer than running a makefile where I always have to look to see what the magic variables are to run what I want.
Mark> 2) ABCL.TEST.ANSI
Mark> In order to support the necessary shenanigans to make the ANSI-COMPILED Mark> and ANSI-INTERPRETED definitions work, we [created a small set of Mark> utility routines][2]. CLEAN-TESTS performs the removal of intermediate Mark> files that the old Makefile used to do. DO-TESTS-MATCHING will run only Mark> those tests whose symbols match a regular expression, which is useful Mark> for repeatedly running a subset of tests.
This also sounds interesting because I often do want to run just a subset.
Mark> 3) ANSI-TEST error "database"
Mark> We [created a simple s-expr "database"][4] for recording test failures Mark> and then [reporting on the results][3]. ABCL is a unique CL Mark> implementation because it is hosted on a JVM for which there are Mark> multiple implementations and whose behavior often varies by operating Mark> system so running the ANSI-TEST suite has proven to be fairly sensitive Mark> to hosting JVM implementation version and operating system. Thus, such Mark> a database of regressions has proven to be of special importance to Mark> ABCL. That being said, my implementation of this idea is pretty Mark> amateurish in hindsight, so I think what is important here is the idea Mark> rather than the implementation.
Again, this sounds pretty cool too. Cmucl has a few extensions that cause the test suite to fail. It would be nice to annotate these test as expected failures. Idealy, a reason for the failure would be part of the annotation. I don't run the test suite that often anymore, but when I do, I always forget exactly which tests are known expected failures and which are new. Having some kind of annotation (other than comments on the test itself) would be awesome!
-- Ray