Attila Lendvai wrote:
That is why I have been suggesting that we provide a test operation that binds a stream --- because most of the test frameworks I have worked with provide a test report, rather than returning results.
fyi, stefil returns a CLOS object containing the test results (and provides slime inspector customizations to present it specially in the inspector).
as it doesn't work too well through asdf:test-op, it also provides a print-object method that prints a minimal text representation of the data.
when test-op'ed, we print the result object to *standard-output*. when used interactively, the test defun that was used to start the testing simply returns the result value which we inspect in slime when needed.
Do you see the same problems I see? I find that my test's output can get interleaved with irrelevant chaff. So some of our code has something like this:
(defmethod perform :around ((op our-test-op) c) (let ((<special variable> (test-stream op))) (call-next-method)))
It's actually far messier than this, because the <special variable>'s package needs to be wrangled, and I needed to do fussy things with make-sub-operation (is there some reason this is not a generic function?) to make sure that our-test-op gets propagated through traverse.
Actually, I tell a lie. Getting make-sub-operation to do the right thing was so hard, that I ended up doing something horrible with CHANGE-CLASS. The resulting code was not suitable for sensitive eyes.