Nikodemus Siivola writes:
2009/8/5 Robert Goldman rpgoldman@sift.info:
Part of the problem with test-op is that the desired behavior has not been specified by the ASDF community. Because of the nature of ASDF, it is impossible for
(asdf:test-system <system>)
to return a value indicating whether or not <system> has passed its tests.
I would propose that as a matter of good style implementations of TEST-OP should signal an error is tests do not pass. It is the simplest way to make sure the result is not misinterpreted...
You wrote "tests do not pass"; just to make sure, the plural was intentional, right? You did /not/ mean it should signal an error for each failing test, did you?
If I'm running the TEST-OP on some system (with the user, not developer, hat on my head), I'd like to get a nice summary about all failing tests to give a sense about the state of the software. I'm not interested in the minutae of the tests themselves.
Signalling an ASDF:TEST-OP-FAILED condition, perhaps even making that hidden in some function, so people can just write (ASDF:TEST-OP-FAIL) within their PERFORM method, is sure an idea if that better fits the architecture of ASDF.
-T.