April 2010 - cffi-devel - mailman3.common-lisp.net

[cffi-devel] Foreign structures by value
by Liam Healy 26 Aug '10

26 Aug '10

I have written Foreign Structures By Value (FSBV) to allow calling foreign functions while passing or returning structures by value. It uses CFFI and libffi. It currently provides two macros for users, fsbv:defcstruct and fsbv:foreign-funcall, which are used exactly like their CFFI counterparts. It works for small test cases I've tried, but probably could use some improvement. If the keepers of CFFI are interested, I would be happy for it to be included in CFFI. In the meantime, it can be found here: http://repo.or.cz/w/fsbv.git Thank you. Liam

2 3

[cffi-devel] Foreign string arguments
by Michael Raskin 22 Aug '10

22 Aug '10

Hello. I use callback feature of CFFI. I want to use an encoding as a default and if I get an invalid string (non-UTF-8 in my case), I'd just reject it by returning an error. Unfortunately, if I declare string arguments with encoding, attempts by CFFI to decode the string raise an error which prevents me from returning protocol-compliant error to the caller. The context: I currently work on FUSE bindings for Common Lisp (or at least SBCL for now). In case of error occurring in my code, I return EIO. Of course, for invalid file names I want to return either EIO or ENOENT. Currently, request just hangs. This can be mitigated by patching CFFI. If I just wrap the conversion in ignore-errors, I lose the distinction between an invalid string and null pointer, but I get a chance to process the request (both cases are wrong in my case). Specifying a designated "wrong string object" and letting user change it seems to be the nicest way: the user gets an error on trying to use the object as a string, but user code can handle it nicely. I changed the following in string conversion code: (defmethod translate-from-foreign (ptr (type foreign-string-type)) (unwind-protect (block nil (handler-case (values (foreign-string-to-lisp ptr :encoding (fst-encoding type))) (error (x) (return t)) )) (when (fst-free-from-foreign-p type) (foreign-free ptr)))) That allowed me to handle invalid requests to the FS by returning ENOENT (just as originally intended). Would you please consider adding some similar modification in official CFFI repository? Michael Raskin

2 1

[cffi-devel] Memory fault using CFFI
by Andrew Kirkpatrick 22 Aug '10

22 Aug '10

Hi, It seems that in many cases when I call code in a library that results in a loaded foreign library function being called, I get a memory fault. Typically this error appears in my slime REPL: Help! 11 nested errors. SB-KERNEL:*MAXIMUM-ERROR-DEPTH* exceeded. 0: ; Evaluation aborted. The 11 errors appear in my *inferior-lisp* buffer, each one similar to: CORRUPTION WARNING in SBCL pid 3239(tid 3039865744): Memory fault at 100020 (pc=0x9c77963, sp=0xb52f6110) The integrity of this image is possibly compromised. Continuing with fingers crossed. Crossed fingers or not, from that point on the lisp image is unstable. It happens when I try to use Elephant to open a BDB store, or when I use CL-SQLITE directly to open a database. However I just discovered that I can successfully run the hello-world.lisp example in CLG. I'm using clbuild on 32bit x86 Ubuntu 9.10, that I used to install SBCL 1.0.35.1 and the latest (according to clbuild) CFFI. Looking closer at CFFI, it fails one test unexpectedly. I'm quite baffled and googling and asking on #lisp draws a blank. The failed test is below. Any ideas? Test DEFCFUN.BFF.2 failed Form: (SUM-127 (MAKE-POINTER 2746181372) (MAKE-POINTER 177623060) -32334.0 3158055028 (MAKE-POINTER 242315091) 4288001754991016425 -21047.0d0 287.0d0 18722 243379286 -8677366518541007140 581399424 -13872 4240394881 1353358999 226 969197676 -26207.0d0 6484 11150 1241680089902988480 106068320 61865 2253 (MAKE-POINTER 866809333) -31613 35616 11715 1393601698 8940888681199591845 (MAKE-POINTER 1524606024) 805638893 3315410736 3432596795 (MAKE-POINTER 1490355706) 696175657106383698 -25438.0 1294381547 26724 (MAKE-POINTER 3196569545) 2506913373410783697 -4405955718732597856 4075932032 3224670123 2183829215657835866 1318320964 -22 -3786.0d0 -2017024146 1579225515 -626617701 -1456 (MAKE-POINTER 3561444187) 395687791 1968033632506257320 -1847773261 48853 142937735275669133 -17974.0 (MAKE-POINTER 2791749948) -14140.0 2707 3691328585 3306.0 1132012981 303633191773289330 (MAKE-POINTER 981183954) 9114.0d0 8664374572369470 -19013.0d0 -10288.0d0 -3679345119891954339 (MAKE-POINTER 3538786709) 23761 -154264605 (MAKE-POINTER 2694396308) 7023 997.0d0 1009561368 241 (MAKE-POINTER 2612292671) 48 1431872408 -32675.0d0 (MAKE-POINTER 1587599336) 958916472 -9857.0d0 111 -14370.0d0 -7308 -967514912 488790941 2146978095 -24111.0d0 13711 86681861 717987770 111 1013402998690933877 17234.0d0 -8772.0 3959216275 -8711 (MAKE-POINTER 3142780851) 9480.0 -3820453146461186120 1616574376 -3336232268263990050 -1906114671562979758 -27925.0d0 9695970875869913114 27033.0d0 1096518219 -12 104 3392025403 -27911 60 89 509297051 -533066551 29158.0 110 54 -9802.0d0 593950442165910888 -79) Expected value: 7758614658402721936 Actual value: 7758614624042983568. Thanks! -- Andrew Kirkpatrick "The first rule of Perl Club... has subscript 0. Unless you changed $[. But don't!" -- blazar on perlmonks.org

2 1

[cffi-devel] ECL specific changes for CFFI
by Juan Jose Garcia-Ripoll 01 May '10

01 May '10

* The former definition for :long-long types in cffi-ecl.lisp was broken. I have agumented ECL with a feature that signals the existence of such a type in ECL and include a patch here for CFFI to take that into account. (Patch attached) * Instead of defining NULL-POINTER-P, it would be better to simply reexport the symbol living in the "EXT" package. Patch attached. * Upon reading the CFFI specification it seems that FOREIGN-FREE can only free memory that has been allocated by CFFI. However the test cases in misc-types.lsp do something else, deallocating the output of my_strdup() explicitely. * At the low level ECL has two different foreign function interfaces: one used in the interpreter and relying on an external library (libffi) and another one, much simpler, using the C compiler. Right now CFFI was only using the former unless it was not available. I provide a patch that chooses the interface depending on the use of the code: interpreter or compiled. (patch attached). * Is there the equivalent of launchpad for CFFI? Should I always submit the patches to the mailing list? I say this because there are other improvements I could forward when I find time. Cheers Juanjo -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

2 6

[cffi-devel] foreign-string-to-lisp should use keywords
by Leslie P. Polzer 01 May '10

01 May '10

A minor nit I found while passing a custom encoding to this function. It's quite cumbersome to have to specify the other optional arguments to this function just to change one of them. Using keywords would also have the benefit of matching the lambda list of lisp-string-to-foreign. Leslie

3 2

Re: [cffi-devel] Latest changes, (probably) no significant errors with CFFI
by Luís Oliveira 29 Apr '10

29 Apr '10

On Thu, Apr 29, 2010 at 8:50 AM, Juan Jose Garcia-Ripoll <juanjose.garciaripoll(a)googlemail.com> wrote: >> CFFI> (macroexpand-1 '(foreign-funcall "foo" :boolean)) >> (NOT (ZEROP (%FOREIGN-FUNCALL "foo" (:INT) ...))) >> >> Or did you mean something else? > > I mean reading and writing data into foreign types, not calling functions. Ah, yes. That too. CFFI> (swank-backend:compiler-macroexpand '(mem-aref p :boolean 4)) (NOT (ZEROP (%MEM-REF P :INT 16))) CFFI> (swank-backend:compiler-macroexpand '(%MEM-REF P :INT 16)) (SB-SYS:SIGNED-SAP-REF-32 P 16) -- Luís Oliveira http://r42.eu/~luis/

1 0

[cffi-devel] New patches: 28-Apr-2010
by Luis Oliveira 29 Apr '10

29 Apr '10

Wed Apr 28 18:56:04 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-ecl: always push no-long-long LONG-LONG is not supported by ECL's interpreter yet. M ./src/cffi-ecl.lisp -1 +1 Wed Apr 28 18:53:37 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-tests: don't use FOREIGN-FREE for malloc()ed memory Patch courtesy of Juan Jose Garcia-Ripoll. M ./tests/libtest.c +6 M ./tests/misc-types.lisp -3 +5 Wed Apr 28 18:42:43 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * New pointer-related tests Testing error situations for some pointer operations. M ./tests/bindings.lisp +4 M ./tests/memory.lisp +36 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-100428.tar.gz Darcsweb URL: http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cffi-cffi;a=summary

1 0

[cffi-devel] Inline expansions for pointer operations
by Juan Jose Garcia-Ripoll 28 Apr '10

28 Apr '10

... provided in this patch. I have a couple of questions. One is whether NULL-POINTER has to cons a new pointer _every_time_. It would be more clever to create a variable and put the pointer there. The other one is whether MAKE-POINTER is used at all, given that we have pointer increment / decrement operations. Juanjo -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

2 4

[cffi-devel] Latest changes, (probably) no significant errors with CFFI
by Juan Jose Garcia-Ripoll 27 Apr '10

27 Apr '10

I have fixed one more bug in ECL, this time related to the alignment of elementary C types (double, int, etc). We now gather the information from the compiler at compilation time, which is great, portable and robust. With this fix CFFI, plus the patches I submitted for the CFFI test suite, ECL has no unexpected failure and the output of the test suite is 4 out of 222 total tests failed: DEFCFUN.NOOP, CALLBACKS.BFF.1, CALLBACKS.BFF.2, STRING.ENCODINGS.ALL.BASIC. There remain two question. The simplest one is as follows: does it make sense to provide inline forms for %foreign-type-size and %foreign-type-alignment, or are they use scarcely? The second question is whether CFFI itself produces inline forms of the expressions dealing with foreign types. I mean conversion to/from lisp, etc. Juanjo -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

1 0

[cffi-devel] Status of CFFI-ECL port
by Juan Jose Garcia-Ripoll 26 Apr '10

26 Apr '10

If we include the fix for the test suite and use the latest changes I incorporated to ECL's git tree, the current status is that there are still 6 unexpected failures 10 out of 222 total tests failed: DEFCFUN.NOOP, CALLBACKS.BFF.1, CALLBACKS.BFF.2, STRING.ENCODINGS.ALL.BASIC, STRUCT.ALIGNMENT.3, STRUCT.ALIGNMENT.4, STRUCT.ALIGNMENT.5, STRUCT.ALIGNMENT.6, STRUCT.ALIGNMENT.7, STRUCT.ALIGNMENT.8. 6 unexpected failures: STRUCT.ALIGNMENT.3, STRUCT.ALIGNMENT.4, STRUCT.ALIGNMENT.5, STRUCT.ALIGNMENT.6, STRUCT.ALIGNMENT.7, STRUCT.ALIGNMENT.8. I hope I will be able to fix them in the coming days. Any hints / help are appreciated. Juanjo -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

1 0

[cffi-devel] New patches: 25-Apr-2010
by Luis Oliveira 26 Apr '10

26 Apr '10

Sun Apr 25 19:12:32 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-ecl: simplify shareable vector implementation Patch courtesy of Juan Jose Garcia-Ripoll. M ./src/cffi-ecl.lisp -28 +1 Sun Apr 25 19:02:19 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-ecl: inline %mem-set and %mem-ref Patch courtesy of Juan Jose Garcia-Ripoll. M ./src/cffi-ecl.lisp -17 +49 M ./tests/memory.lisp +5 Sun Apr 25 18:50:04 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-ecl: use C-INLINE when producing compiled code Using the ECL extension EXT:WITH-BACKEND, CFFI can produce code that works differently when using the interpreter than when using the lisp2C compiler. This leads to more efficient code. This change is backwards compatible. Patch courtesy of Juan Jose Garcia-Ripoll. M ./src/cffi-ecl.lisp -22 +41 Sun Apr 25 18:39:14 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-ecl: reexport NULL-POINTER-P from SI Patch courtesy of Juan Jose Garcia-Ripoll. M ./src/cffi-ecl.lisp -4 +1 Sun Apr 25 18:37:48 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * cffi-ecl: use ECL's :long-long feature Patch courtesy of Juan Jose Garcia-Ripoll. M ./src/cffi-ecl.lisp -3 +5 Sun Apr 25 08:24:43 EDT 2010 Luis Oliveira <loliveira(a)common-lisp.net> * Allegro: fix WITH-FOREIGN-POINTER - Fixes previous patch. - New test: WITH-FOREIGN-POINTER.CONSTANT-SIZE. M ./src/cffi-allegro.lisp -2 +2 M ./tests/memory.lisp -2 +11 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-100425.tar.gz Darcsweb URL: http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cffi-cffi;a=summary

1 0

[cffi-devel] Updating ECL's implementation of shareable vectors
by Juan Jose Garcia-Ripoll 25 Apr '10

25 Apr '10

ECL already provides a function to get a pointer associated to an array. CFFI should use it. Patch attached. -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

2 1

Re: [cffi-devel] %mem-{set,ref} can be inlined?
by Luís Oliveira 25 Apr '10

25 Apr '10

On Sun, Apr 25, 2010 at 5:49 PM, Juan Jose Garcia-Ripoll <juanjose.garciaripoll(a)googlemail.com> wrote: > Ok, if this is the case then here is a slightly simpler patch. [...] > Could you add it to the test suite, just in case? Pushed your patch, along with the test. Thanks. -- Luís Oliveira http://r42.eu/~luis/

1 0

[cffi-devel] %mem-{set,ref} can be inlined?
by Juan Jose Garcia-Ripoll 25 Apr '10

25 Apr '10

I had a look at the definitions of these two functions. Right now in cffi-ecl.lisp it seems that they are only called with basic types, for they call cffi-type->ecl-type which only deals with things like integers, doubles, etc. Could I just produce a compiler macro that relies on this? The result would be much much simpler than this as it could be inlined and make use of ECL's embedded C statements. Actually it would be nice if instead of a compiler macro I could define them to be just plain macros. Anything that prevents it? Juanjo (defun %mem-ref (ptr type &optional (offset 0)) "Dereference an object of TYPE at OFFSET bytes from PTR." (let* ((type (cffi-type->ecl-type type)) (type-size (ffi:size-of-foreign-type type))) (si:foreign-data-ref-elt (si:foreign-data-recast ptr (+ offset type-size) :void) offset type))) -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

2 3

[cffi-devel] Strange equivalence
by Juan Jose Garcia-Ripoll 25 Apr '10

25 Apr '10

In the file cffi-ecl.lisp of I find the equivalence table (:char :byte "char") (:unsigned-char :unsigned-byte "unsigned char") that associates, for the ECL port, a byte with the CFFI :char type. Is this intended per-specification? Why not using ECL's :char type, which might be more than one byte if the compiler and operating system expect so? Juanjo -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

2 1

[cffi-devel] Not able to use constant variables in with-foreign-pointer
by Jianshi Huang 25 Apr '10

25 Apr '10

Hi, There's a minor problem in the cffi-allegro.lisp that prevents using constant variables in with-foreign-pointer. Would you please apply the patch? Tue Apr 20 19:16:26 JST 2010 Jianshi Huang <huang(a)msi.co.jp> * Fix bug that size in with-foreign-pointer cannot be a constant variable. diff -rN old-cffi/src/cffi-allegro.lisp new-cffi/src/cffi-allegro.lisp 139c139 < (ff:with-static-fobject (,var '(:array :char ,size) --- > (ff:with-static-fobject (,var (list :array :char ,size) Cheers, Jianshi

3 3

[cffi-devel] New patches: 21-Apr-2010
by Luis Oliveira 22 Apr '10

22 Apr '10

Wed Apr 21 05:19:56 EDT 2010 Jianshi Huang <huang(a)msi.co.jp> * Allow the use of constants in WITH-FOREIGN-OBJECT for Allegro. M ./src/cffi-allegro.lisp -1 +1 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-100421.tar.gz Darcsweb URL: http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cffi-cffi;a=summary

1 0

[cffi-devel] About double pointers and C & operator
by Alexandre Paes 19 Apr '10

19 Apr '10

Hi, For a test project, i was creating a CFFI interface to the OpenCV library and my limited knowledge of CFFI has left me wondering how to achieve something. According to the documentation on the OpenCV wiki, in order to start capturing from a webcam one would call the following function: CvCapture* cvCreateCameraCapture (int index) The CvCapture is an opaque structure without public interface, so i simply store the pointer returned by the function. To end the image capturing, one would release the capture through the function: void cvReleaseCapture(CvCapture** capture) On the OpenCV wiki there is a sample program ( http://opencv.willowgarage.com/wiki/CameraCapture) that basically has: int main() { ... CvCapture* capture = cvCaptureFromCAM( CV_CAP_ANY ); ... cvReleaseCapture( &capture ); ... } I've defined the foreign functions as: (cffi:defcfun ("cvCreateCameraCapture" create-camera-capture) :pointer (index :int)) (cffi:defcfun ("cvReleaseCapture" %release-capture) :void (capture :pointer)) Then in my program i call 'create-camera-capture and it works, the webcam lights up and starts capturing. I even managed to simulate the &capture behavior by renaming the previous declaration of release-capture to %release-capture and creating a new definition like: (defun release-capture (capture*) (cffi:with-foreign-object (capture** :long) (setf (cffi:mem-ref capture** :long) (cffi:pointer-address capture*)) (%release-capture capture**))) It works but it doesn't feel all that nice, is this the best/correct way of achieving the same as the C code? Cheers, Alex Paes

2 1