On Sat, Apr 17, 2010 at 10:29 PM, Robert Goldman rpgoldman@sift.infowrote:
I applaud the idea of having a declarative spec of tests to be run, but is the test function really the only way to do this? Seems like there are test frameworks where what you want to do is to specify (somehow) a set of test objects to be run, where the framework "knows" how to run them.
There are several reasons why I used the function name and did not create framework specific options
* People are using their own custom functions for running tests, and never in an uniform way. * It was argued here that ASDF should never be bound to any particular test framework. I have become convinced of the arguments in that respect. * We can not impose test frameworks how to behave, but we can recommend users what they should output their tests (please have a look at the thread I started long ago).
suggestion: if we are to propose an ASDF extension for testing, we should first survey the CL test frameworks (off-hand: FiveAM, NST, cl-unit, LIFT, RT, and ? ...) and see what they would most "like" in the way of a test specification.
Given that the :tests keyword work, I have a mean to do what you want without really binding ASDF to the test framework.
The idea would be to provide plug-ins for those frameworks with system classes that take those new options. We would then extend the syntax of :tests to include closures (would be great!) and let the system-test class set the appropriate value of :tests.
But note that is completely independent of the current patch and does not preclude or make it less useful, since users may also want to write their own tests.
But what this does bring into the light is the need of means to describe that a given DEFSYSTEM form depends on a given plug in or other system. This would be what I was calling the :ASDF-DEPENDENCIES or :ASDF-SUPPORT option in previous emails. Something like
(defsystem :my-test-system :asdf-dependencies (:rt-system) :rt-tests ("TEST1" "TEST2" ...) ...)
Let's see if I can produce something like that tonight -- stranded in Madrid because of the volcano cloud :-/
Juanjo