On Mon, Oct 19, 2009 at 12:00 AM, Robert Goldman rpgoldman@sift.info wrote:
In particular because it's my impression that the problems have been over-stated.
I think so. And this is indicated by the following paragraphs
- Not all regression test frameworks are functional, returning values.
Some write reports instead.
That does not mean anything. ASDF may impose one behavior w.r.t to the output and the package maintainer may decide to output T saying "hey, there might be errors, but I do not know." Later on other software and test suites will catch up. With RT, for instance it is trivial to grab the number of failed tests.
- One needs to come up with a means of combining operation results
that takes into account the structure of the plan that traverse produces and that operate then executes. If in order to test-op X, I must test-op A and B, how do I combine together the test results from A, B, and X into a top-level operation result. Perhaps OPERATION-ANCESTOR can be pressed into service.
You are just imposing too much complexity. If I want to test package Cl-UNICODE, I do nont want to test FLEXI-STREAMS or U-SOCKETS. Tests should be atomic and not generate a tree of actions like ASDF does not for everything.
- Is there some way to do this such that the regression-testing
framework need not be made aware of ASDF?
No, there is none. Regression tests are libraries which are used by people using ASDF. Eventually they may provide their own operations, extending what ASDF provides, but currently it is better to have some bare minimum from ASDF and let each package maintainer decide how they compute the success of their test suite.
Blocking this development just because there are 5 test suites and you do not know how to combine them with ASDF is really absurd. ASDF's specifications can not depend on what your company or other people's companies are setting up for their workflow.
Juanjo