Hi,
I had some time to work on the port of CFFI to ECL and I have attached most of the fixes. They include
+ Fixed the port to architectures without dynamical FFI. (In these ports CFFI foreign function calls only work when lisp code is compiled)
+ ECL supplies its own version of RT hence this need not appear in the ASDF cffi-test.
+ Tests with too large number of arguments (> 34) are marked as expected to fail, since ECL's FFI does not support such function calls.
I had to play a few tricks because CFFI does not allow us to include headers function declarations and ECL cannot rely on the arguments to FOREIGN-FUNCALL which are most of the time bogus. For instance, :string is an alias to :pointer and when you call (foreign-funcall "strlen" :string .. ) we cannot produce a proper C declaration to strlen that does not collide with the one provided by the header <string.h>.
So in the end I end up always using the routine dlsym() to discover the functions and coerce them to a function pointer created with the information given by CFFI. The outcome is therefore much less efficient than what ECL can potentially do.
Regards
Juanjo
On 10/11/06, Juan Jose Garcia-Ripoll jjgarcia@users.sourceforge.net wrote:
I had some time to work on the port of CFFI to ECL and I have attached most of the fixes.
Thanks, I have pushed the patch.
- ECL supplies its own version of RT hence this need not appear in the
ASDF cffi-test.
Minus the RT changes that would break Lispworks and we prefer to use the asdf-installable RT for all Lisps anyway. (Hmm, I think we even had a good reason, though I can't quite remember.)
2006/10/13, Luís Oliveira luismbo@gmail.com:
On 10/11/06, Juan Jose Garcia-Ripoll jjgarcia@users.sourceforge.net wrote:
- ECL supplies its own version of RT hence this need not appear in the
ASDF cffi-test.
Minus the RT changes that would break Lispworks and we prefer to use the asdf-installable RT for all Lisps anyway. (Hmm, I think we even had a good reason, though I can't quite remember.)
Why does a change prefixed by #+ecl / #-ecl affect other implementations?
Also you are forcing people to install a version of RT which is basically the one with ECL, though ours output more useful information, such as the message of a condition when the output is one.
Anyway, when I find time I will finish the support for statically compiled callbacks.
Juanjo
On 10/12/06, Juan Jose Garcia-Ripoll jjgarcia@users.sourceforge.net wrote:
Why does a change prefixed by #+ecl / #-ecl affect other implementations?
Sorry, I wasn't clear. The particular change that breaks Lispworks is this one:
- (let ((rtest::*compile-tests* compiled) + (let ((rt::*compile-tests* compiled)
Also you are forcing people to install a version of RT which is basically the one with ECL, though ours output more useful information, such as the message of a condition when the output is one.
Good point. James, do you remember why we've resisted to use other versions of RT so far? :-)
Anyway, when I find time I will finish the support for statically compiled callbacks.
Cool, thanks!
On Thu, 2006-10-12 at 23:27 +0100, Luís Oliveira wrote:
Also you are forcing people to install a version of RT which is basically the one with ECL, though ours output more useful information, such as the message of a condition when the output is one.
Good point. James, do you remember why we've resisted to use other versions of RT so far? :-)
I'm fine with using ECL's version of RT if it's advertised as a public interface for general use. IIRC, the SBCL maintainers preferred to reserve the right to change SB-RT incompatibly, as its really intended for testing of contribs, not for user code.
James
On 10/12/06, James Bielman jamesjb@haytonsystems.com wrote:
IIRC, the SBCL maintainers preferred to reserve the right to change SB-RT incompatibly, as its really intended for testing of contribs, not for user code.
Ah, right. The only problem now is that ECL's RT package's name is SB-RT and the only nickname it has is RT so we can't use it conveniently anyway.
2006/10/13, Luís Oliveira luismbo@gmail.com:
On 10/12/06, James Bielman jamesjb@haytonsystems.com wrote:
IIRC, the SBCL maintainers preferred to reserve the right to change SB-RT incompatibly, as its really intended for testing of contribs, not for user code.
Ah, right. The only problem now is that ECL's RT package's name is SB-RT and the only nickname it has is RT so we can't use it conveniently anyway.
This is no longer true as of CVS. Otherwise I would not have been able to run the tests using ECL's RT.
Juanjo
On 10/13/06, Juan Jose Garcia-Ripoll jjgarcia@users.sourceforge.net wrote:
Ah, right. The only problem now is that ECL's RT package's name is SB-RT and the only nickname it has is RT so we can't use it conveniently anyway.
This is no longer true as of CVS. Otherwise I would not have been able to run the tests using ECL's RT.
Oh, ok. Why the change from RTEST to RT then? Ah, I see it has REGRESSION-TEST as a nickname, I'll use that.
On Wed, 2006-10-11 at 22:20 +0200, Juan Jose Garcia-Ripoll wrote:
I had to play a few tricks because CFFI does not allow us to include headers function declarations and ECL cannot rely on the arguments to FOREIGN-FUNCALL which are most of the time bogus. For instance, :string is an alias to :pointer and when you call (foreign-funcall "strlen" :string .. ) we cannot produce a proper C declaration to strlen that does not collide with the one provided by the header <string.h>.
So in the end I end up always using the routine dlsym() to discover the functions and coerce them to a function pointer created with the information given by CFFI. The outcome is therefore much less efficient than what ECL can potentially do.
In a way, this actually seems like the right thing to me. I think this is roughly what the Lisps that compile directly to native code are doing, except they only perform the dlsym once (via linkage tables or somesuch).
Would it be possible to look up the function address in a LOAD-TIME-VALUE form (not knowing how L-T-V works in compiled ECL code...) or otherwise caching it?
James
(defun %mem-ref (ptr type &optional (offset 0)) "Dereference an object of TYPE at OFFSET bytes from PTR."
- (let* ((type (convert-foreign-type type))
- (let* ((type (cffi-type->ecl-type type)) (type-size (ffi:size-of-foreign-type type))) (si:foreign-data-ref-elt (si:foreign-data-recast ptr (+ offset type-size) :void) offset type)))
(defun %mem-set (value ptr type &optional (offset 0)) "Set an object of TYPE at OFFSET bytes from PTR."
- (let* ((type (convert-foreign-type type))
- (let* ((type (cffi-type->ecl-type type)) (type-size (ffi:size-of-foreign-type type))) (si:foreign-data-set-elt (si:foreign-data-recast ptr (+ offset type-size) :void)
Why in %mem-ref/%mem-set is used (+ offset type-size) instead of (* offset type-size)?
Or I do not understand something?
Thanks!
2006/10/15, Yaroslav Kavenchuk kavenchuk@tut.by:
Why in %mem-ref/%mem-set is used (+ offset type-size) instead of (* offset type-size)?
I don't know. It looks like a bug, but this is a part of cffi-ecl.lsp which I still have not revised: I only changed convert-foreign-type to cffi-type->ecl.type
Juanjo
On Mon, 2006-10-16 at 00:22 +0200, Juan Jose Garcia-Ripoll wrote:
2006/10/15, Yaroslav Kavenchuk kavenchuk@tut.by:
Why in %mem-ref/%mem-set is used (+ offset type-size) instead of (* offset type-size)?
I don't know. It looks like a bug, but this is a part of cffi-ecl.lsp which I still have not revised: I only changed convert-foreign-type to cffi-type->ecl.type
I don't understand exactly what SI:FOREIGN-DATA-RECAST is for here, but the OFFSET argument to %MEM-REF is a raw byte offset, not an element count, so adding it to the base pointer instead of multiplying it by the size seems like a reasonable thing to do here.
Luís, it looks like this was added in the patch "Updated ECL Support" about a year ago---do you remember why?
James
On 10/16/06, James Bielman jamesjb@haytonsystems.com wrote:
I don't understand exactly what SI:FOREIGN-DATA-RECAST is for here, but the OFFSET argument to %MEM-REF is a raw byte offset, not an element count, so adding it to the base pointer instead of multiplying it by the size seems like a reasonable thing to do here.
Luís, it looks like this was added in the patch "Updated ECL Support" about a year ago---do you remember why?
That patch was sent in by Michael Goffioul.
2006/10/16, James Bielman jamesjb@haytonsystems.com:
I don't understand exactly what SI:FOREIGN-DATA-RECAST is for here, but the OFFSET argument to %MEM-REF is a raw byte offset, not an element count, so adding it to the base pointer instead of multiplying it by the size seems like a reasonable thing to do here.
Ok. Then it is correct. After sending my previous email I wondered why it would be wrong, given that ECL passes most tests.
Juanjo
-
James Bielman -
Juan Jose Garcia-Ripoll -
Luís Oliveira -
Yaroslav Kavenchuk