[cffi-devel] result reporting for random numeric tests in tests/random-tester.lisp
I'm almost finished with a random test generator in tests/random-tester.lisp. However, I'd like to know how you would like results to be reported by the main driver function, `run-random-numeric-tests'. Shall it print RT test names and expect you to analyze the temporary C/Lisp files? Shall it print type signatures as well? Shall it try to find what types the failed tests have in common? I'd like to provide all convenient information without overengineering any further than I already may have. There is also the possibility that I have misinterpreted how "random" should be parameterized with regard to tests. Everything I have so far: http://scompall.nocandysw.com/cffi/randtest-0.darcs.patch is a darcs patch http://scompall.nocandysw.com/cffi/randtest-0.diff is `darcs whatsnew' output on the file log: A random numeric test generator (except outermost driver code) * Everything on pages "Temporary file generation" and "Test driver and result handling", and most of "Random test selection" is new. * Add a "stream" argument to C-FUNCTION and C-CALLBACK. * `output-c-code' is broken, and I don't know whether it's useful, so snap it in half more permanently. -- Stephen Compall http://scompall.nocandysw.com/blog
Stephen Compall wrote:
Shall it print RT test names and expect you to analyze the temporary C/Lisp files? Shall it print type signatures as well?
Implemented this. It works, but I get plenty of failed tests, and keeping the tempfiles on test failure doesn't quite work; I will work out something for that, possibly by removing use of do-tests. http://scompall.nocandysw.com/cffi/randtest-1.darcs.patch is a darcs patch Thu Aug 31 04:34:08 CDT 2006 Stephen Compall <scompall@nocandysw.com> * Make working driver for run-random-numeric-tests * Rename `combinations' to `permutations', because that's what they really are. * `gen-function-test' and `gen-callback-test' now answer the RT test name (a symbol) as their second values. * *test-c-header* and *library-symbol-name*: new variables. * random-range: Off-by-one error. * c-function: missing open paren for void*->unsigned-int cast. * may-be-gentestname-p didn't really work as was, so now using new helper string-starts-with-p. * Rename write-test-pair to prepare-test-pair, and split it up. It also answers its RT test names. * Bleeding useful new helper `shlib-target-name'. * prepare-make-target, write-*-preamble: New functions. -- Stephen Compall http://scompall.nocandysw.com/blog
On 8/31/06, Stephen Compall <s11@member.fsf.org> wrote:
Implemented this. It works, but I get plenty of failed tests, and keeping the tempfiles on test failure doesn't quite work; I will work out something for that, possibly by removing use of do-tests.
Cool stuff. Keep up the good work. :-) -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/
participants (2)
-
Luís Oliveira -
Stephen Compall