Part of the problem with test-op is that the desired behavior has not been specified by the ASDF community. Because of the nature of ASDF, it is impossible for
(asdf:test-system <system>)
to return a value indicating whether or not <system> has passed its tests.
FWIW, I recently changed operate-on-system (and therefore asdf:test- system, asdf:load-system, etc.) to return the operation instance that they processed.
If ASDF also added and documented (shocking!) a few canonical slots in the test-op class then these could be used to transmit information. As a first pass, I'd suggest
test-properties - a property list of test system specific stuff test-result - a keyword from the list :success, :failures, :errors, errors-and-failures
Some time ago, I proposed that ASDF provide a stream argument to the test-op, providing a place into which the testing tool could dump its test report for human inspection. I can't say that this suggestion received universal approbation. Or disapprobation. It merely received near-universal lack of interest! ;-)
Sigh. I think this is a good idea; if we added a test-stream slot to the test-op, then test systems could find it there.
A tricky part is that test systems may not want to rely on the existence of ASDF while running. So they'll need some sort of adaptor that adds stuff to the ASDF operation if it exists. Would it make sense to define and export *current-operation* to make this easier?
Related to the question of what the test-op should provide to its invoker is the question of how dependencies should be propagated.
I think that the default is that test-op depends on load-op.
other thoughts? -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM * gwking on twitter