On Tue, Jan 10, 2012 at 12:00 PM, Anton Vodonosov avodonosov@yandex.ru wrote:
What meaning do you put into the term "expected failure"? Does it mean the library is buggy, but these bugs are known? Or it means that some non-required features are absent, but the library in general OK?
I may have not been consistent in my usage of marking expected failures, but they mark known bugs unlikely to be fixed in the short-term. Either in CFFI or in the Lisp implementation. Now that we have a bugtracker, known failures should definitely point to their respective issue.
Non-required features are marked using cffi-sys::foo symbols in *features* so we shouldn't have failures related to that.
I am interested in both short answer - as a library author, how do you think CFFI test suite should be marked if only expected failures present - OK or FAIL?
Depends on the use case. In terms of notifications, I would rather be warned about new failures. In terms of a summary, I'd like to see the results broken down into OK, FAIL, KNOWNFAIL.
And also I am curious in this concrete example, what these 4 failures mean for CFFI on ECL?
DEFCFUN.NOOP fails because ECL's :void type returns NIL instead of (VALUES) and we'd need to implement a work-around in CFFI. Since this is a such a minor detail, it's sitting unfixed in the foreseeable future. CALLBACKS.BFF.1 is an ECL bug: it used to crash (or hang?) while compiling the SUM-126-NO-LL defcallback. It might have been fixed in the meantime. The string encoding failures are a known CFFI bug, IIRC.
HTH,