Anton Vodonosov avodonosov@yandex.ru writes:
- success should also be signaled, so we can distinguish a version where this new protocol is not implemented from the version where tests pass
That is a good idea.
For a caller of asdf:test-op it would be more convenient to have a single signal. Ideally, it should be just a return value of the asdf:operate function, as I understand we only consider the possibility of test result being signaled multiple times during test-op because we hope to make it work for everyone without library authors explicitly modify their code, but adding this new functionality to test frameworks. A good goal, although I can imaging some corner cases. Still, even if we expect test results being signalled multiple times during a test-op, it would be good to provide a wrapper which aggregates them into a single return value.
(common-test-results:collect (asdf:test-system "my-system"))
That is a good idea. I think it goes together well with the fully qualified test names recommendation.
- as others mention, to me it also occurred this new functionality should not necessarily be declared inside of ASDF, it could be some separate library, say common-test-result. I'm not 100% sure about this, but currently, lean more towards separate lib, at least for the beginning. ASDF test-op docs could just referer to it.
Raising a signal is a work-around for the inability of TEST-OP to return a result. I would like to avoid making an entire library out of a work-around that is specific to ASDF.
-- Vladimir Sedach Software engineering services in Los Angeles https://oneofus.la