February 2006 - cffi-devel - mailman3.common-lisp.net

Re: [cffi-devel] New patches: 3-Feb-2006: x86-64
by Hoehle, Joerg-Cyril 10 Feb '06

10 Feb '06

James Bielman wrote: >> Could you please confirm my supposition that long-long works out of >> the box with clisp on your amd64 machine? >How would I go about testing this? I don't see a LONG-LONG type in >the FFI package (or do I need a CVS version? I'm using the latest >release). Map cffi:long-long to ffi:sint64 and ffi:uint64. I have not added an ffi:long-long type. Not ideal, but good enough. I've now updated CVS CLISP to support interfacing to a 64bit "long long" entity on all platforms where some 64bit type is available (e.g. gcc, MS-VC). In other words, cffi-features:no-long-long should be removed from cffi-clisp (or you may wish to conditionalize on the version number or whatever, but that's too restrictive since e.g. I'm convinced that ffi:uint64 has always worked on 64bit machines, where long=long long)). Please test & report if it fails. Furthermore, I think you need to think about what CFFI should do e.g. on the MS-VC platform. It does not have a "long long", it's called __int64 instead. But this is a 64bit type (and possibly some system structures or libraries use that, I don't know). CLISP's ffi:uint64 works with __int64. Actually, given my tests, I'd not recommend trying to call MS-VC __int64 receiving or returning functions via gcc long long or vice-versa. They may have incompatible calling conventions (speculation, but MS-VC6 has a different one from GCC4 on Linux/i686. I haven't tested cygwin/mingw). Pointers to such 64bit ints are no problem at all. Regards, Jorg Hohle.

2 1

[cffi-devel] make Texinfo @r and colorize play nice
by Stephen Compall 10 Feb '06

10 Feb '06

http://csserver.evansville.edu/~sc87/cffi/atr-and-colorize.darcs.patch Thu Feb 9 21:50:38 CST 2006 Stephen Compall <scompall(a)nocandysw.com> * make Texinfo @r and colorize play nice - Comment out master menu entry for explain-foreign-slot-value so manual builds again. - Add @r to all Lisp comments in manual. - (Tutorial-easy_setopt): Revert previous @dots{}; it is literal `...', not an ellipsis. Oops. - In colorize-lisp-examples.lisp, remove the output generated by @r from Lisp blocks. Also shove around the process-file code some. -- Stephen Compall http://scompall.nocandysw.com/blog

2 3

[cffi-devel] New patches: 9-Feb-2006
by loliveira＠common-lisp.net 10 Feb '06

10 Feb '06

Thu Feb 9 20:11:17 CST 2006 Luis Oliveira <loliveira(a)common-lisp.net> * Remove @r{} in manual's code comments (at least until the colorize script can cope with these) M ./doc/cffi-manual.texinfo -2 +2 Thu Feb 9 18:57:07 CST 2006 Luis Oliveira <loliveira(a)common-lisp.net> * More minor doc changes. - @emph -> @var. - change var's style to something more sensible in the CSS file. - remove (setf callback) from the docs. - fix some typos. M ./doc/cffi-manual.texinfo -104 +102 M ./doc/style.css -1 +2 Thu Feb 9 18:36:50 CST 2006 Luis Oliveira <loliveira(a)common-lisp.net> * Make :boolean accept typedefs to integer types. M ./src/types.lisp -1 +1 Thu Feb 9 16:51:09 CST 2006 Luis Oliveira <loliveira(a)common-lisp.net> * Minor documentation changes - Have the Makefile's clean target delete some more of the temporary texinfo files. - Comment out reference to explain-foreign-slot-value. M ./doc/Makefile -2 +2 M ./doc/cffi-manual.texinfo -1 +1 Thu Feb 9 12:39:38 CST 2006 Stephen Compall <scompall(a)nocandysw.com> * reference prose in Pointers and Strings manual chapters - (Tutorial-easy_setopt): Use @dots{}. - (seealso): Remove old comment about @code formatting. - Pointers chapter: Describe the theory of foreign data, dealing with pointers per se, and expand on the allocation description. - (Strings): A short paragraph by way of introduction. M ./doc/cffi-manual.texinfo -10 +56 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-060209.tar.gz Darcsweb URL: http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cffi;a=summary

1 0

[cffi-devel] short patch to Pointers and Strings chapters
by Stephen Compall 09 Feb '06

09 Feb '06

http://csserver.evansville.edu/~sc87/cffi/cffi-manual-4s11.darcs.patch Thu Feb 9 12:39:38 CST 2006 Stephen Compall <scompall(a)nocandysw.com> * reference prose in Pointers and Strings manual chapters - (Tutorial-easy_setopt): Use @dots{}. - (seealso): Remove old comment about @code formatting. - Pointers chapter: Describe the theory of foreign data, dealing with pointers per se, and expand on the allocation description. - (Strings): A short paragraph by way of introduction. -- Stephen Compall http://scompall.nocandysw.com/blog

2 1

Re: [cffi-devel] Problem with SDL_Init using CFFI with Clisp in W indows 98
by Hoehle, Joerg-Cyril 09 Feb '06

09 Feb '06

Hi, Jeremy Smith wrote: >CFFI-based SDL >module does not work on Windows 98 under Clisp. I just tested it on my >Windows 98 laptop (P166 :-) and it says the same thing. "FFI: >FOREIGN-LIBRARY-FUNCTION: No dynamic object named 'SDL_Init' in library >:DEFAULT" I've a different opinion from James :) The one difference I see is that your CLISP code explicitly mentions the library to look after. That is the most portable way and the way I recommend, even for UNIX. It's precise, and you won't get hit by name clashes (e.g. Yaroslav Kavenchuk found duplicate sprintf across MS-Windows .dlls, one crashes, the other not... Which one you get depends on the order in which the system opened libraries, AFAIK it's not even controlled by CLISP.) CFFI does not provide such a mechanism. There has been some recent work in that area, e.g. define-foreign-library and use-foreign-library. But it still seems (or my darcs copy is too old) that CFFI is not at the point where it can say for each function, or at the top of a file defining functions, "this/these belong/s to library XYZ". Please correct me if I'm wrong and this is antinews. W.r.t. CLISP: o In MS-Windows >98, constantly using :library :default should work (clisp will scan all open libraries, you bet this takes time). o On Linux, CLISP was patched last year to make :library :default so that it would scan all libraries (to help people (e.g. CFFI users) *and* to work around bugs in other software, e.g. incomplete library dependencies). o Doesn't Frank Buss got a patch to make MS-Windows-98 work (I lost track of it)? However, if MS-Windows-98 does not have means to inspect dlls for symbols (e.g. psapi.dll), then I don't see how this can be patched. You can try to find a psapi.dll for MS-windows98?? You could only modify CFFI to emit a (:library *my-library*) into the code from defcfun and the above mentioned cffi:use-foreign-library to set that variable. IMHO, the problem is with CFFI, not CLISP. Lispworks would also benefit from using module names. My bet/hope is that this is just a matter of time and volunteers to write code, and CFFI will behave nicely :-) Modules and the like have already been discussed in this list, here and there. But we need people like you to complain to make discussions progress to real code! Regards, Jorg Hohle.

5 5

Re: [cffi-devel] must cffi-sys::with-pointer-to-vector-data go?
by Hoehle, Joerg-Cyril 08 Feb '06

08 Feb '06

rif (Raymund IIRC?) [rif(a)MIT.EDU] wrote: >I'm not sure I fully understand your example or concern. There's a .sig along "Common Lisp is for the things that work." >Obviously, if my program is multithreaded or the C program needs to >maintain the state across calls or I don't know what else, then the >semantics become very different. [...] >documentation caveats that it's not going to DTRT across all platforms >if you require true sharing? Calling something "shareable vector" and recognizing it's not true sharing is not something that works. Does this make my position clearer? There's IMHO already too much broken SW and APIs out there in the world (Lisp, Java, Python, C++, c#, C, whatever). We should try to focus on a) things that work b) things that work at large c) things that work reliably The environment API descibed in CLtL2 was removed from ANSI-CL. Why? because it was found to be subtly broken. No more reason than that. And I showed that it's blatantly broken to try to mimic shared vectors with copying. Do you see the parallels? In summary, I'm not against an optional shared-vector API at all. It may even be a distinguishing feature of CFFI vs. UFFI that it provides many optional APIs. But I'm really opposed to broken things, e.g. trying to provide/implement the shared-vector API using copying. Please strive for correctness, not brokeness. Regards, Jorg Hohle.

1 0

Re: [cffi-devel] New patches: 3-Feb-2006: x86-64
by Hoehle, Joerg-Cyril 08 Feb '06

08 Feb '06

James, >Fri Feb 3 02:29:54 CST 2006 James Bielman <jamesjb(a)jamesjb.com> > * Add CFFI-FEATURES:X86-64 and push it on *FEATURES* on AMD64 Lisps. > - Currently this is supported under SBCL and CLISP. Could you please confirm my supposition that long-long works out of the box with clisp on your amd64 machine? I'm in the process of making long-long generally work in clisp, and I need a little input. So far, I'd presume the 64bit machines are not disturbed by the 10 year old long long code in clisp (because there, long = long long = 64bit, long-long is not used). ffi:uint64 conversion raising an error in the MS-VC builds is a different error. It was sort of a preprocessor confusion (HAVE_LONGLONG vs HAVE_LONG_LONG). It should work now (but I haven't tested CVS with MSVC for the last 6 month). I tested CLISP with MSVC about ffi:uint64 by backporting a one-line change to my 6 month old MSVC build tree. Is there anybody listening using MS-VC with CLISP? Please contact me. Thanks, Jorg Hohle.

2 1

[cffi-devel] optimization opportunities for type translators
by Hoehle, Joerg-Cyril 08 Feb '06

08 Feb '06

Hi, James Bielman wrote: >Thu Feb 2 18:41:28 CST 2006 James Bielman <jamesjb(a)jamesjb.com> > * Add an optimization for defining non-translatable types. Is that something akin to sealing? Maybe that's a good step towards optimization. I've noticed the following behaviour (with CFFI-CVS from last week or so), which led me to write this note: Since the addition of the new type translators, I'd got some bad feelings in my stomach (and wrote about it), without the time to look closer. Now I looked. Somehow I feel there's a missing API for compile-time transformations in the new branch, similar to the old duality provided by translate-to-c <-> to-c-FORM. To make a long story short, here's output for test struct.5 (defctype my-int :int) (defcstruct s5 (a my-int)) (deftest struct.5 (with-foreign-object (s 's5) (setf (foreign-slot-value s 's5 'a) 42) (foreign-slot-value s 's5 'a)) 42) The old CFFI produces (just skim over the rightmost column): 12 Bytecode-Instruktionen: 0 (LOAD&PUSH 1) 1 (CALL1&PUSH 0) ; FFI:FOREIGN-ADDRESS 3 (CONST&PUSH 1) ; 42 4 (LOAD&PUSH 1) 5 (CONST&PUSH 2) ; FFI:INT 6 (CONST&PUSH 3) ; 0 7 (CALL 4 4) ; FFI::WRITE-MEMORY-AS 10 (LOAD&PUSH 0) 11 (CONST&PUSH 2) ; FFI:INT 12 (CONST&PUSH 3) ; 0 13 (CALL 3 5) ; FFI:MEMORY-AS 16 (SKIP&RET 3) Neat. Perfect. The new branch of CFFI produces: 18 Bytecode-Instruktionen: 0 (LOAD&PUSH 1) 1 (CALL1&PUSH 0) ; FFI:FOREIGN-ADDRESS 3 (CONST&PUSH 1) ; 42 4 (CONST&PUSH 2) ; #<CFFI::FOREIGN-TYPEDEF MY-INT> 5 (CALL2&PUSH 3) ; CFFI::TRANSLATE-TYPE-TO-FOREIGN 7 (LOAD&PUSH 0) 8 (LOAD&PUSH 2) 9 (CONST&PUSH 4) ; FFI:INT 10 (CONST&PUSH 5) ; 0 11 (CALL 4 6) ; FFI::WRITE-MEMORY-AS 14 (SKIP 1) 16 (LOAD&PUSH 0) 17 (CONST&PUSH 4) ; FFI:INT 18 (CONST&PUSH 5) ; 0 19 (CALL&PUSH 3 7) ; FFI:MEMORY-AS 22 (CONST&PUSH 2) ; #<CFFI::FOREIGN-TYPEDEF MY-INT> 23 (CALL2 8) ; CFFI::TRANSLATE-TYPE-FROM-FOREIGN 25 (SKIP&RET 3) Ouch. Two generic function calls, which each call another generic function (e.g. translate-to-foreign), some of which cons, e.g. (defmethod translate-type-to-foreign (value (type foreign-typedef)) ;;We build a list out of the second value returned... ;;IMHO bug: 1. undocumented and 2. most don't provide a second value So we compare o 4 GF calls in total, and some consing o to nothing like this previously o with something a C compiler compiles to 2-4 native instructions. IMHO this hurts. It hurts even more so, since CFFI originaly started out from disatisfaction with UFFI on cmucl, where James Bielman observed boxing of float values IIRC, and he initially reported performance improvements over UFFI using his approach. Right now, I'd expect CFFI to lag far behind UFFI in performance (at least with typedefs and structs, other tests are still compiled fine). The type translators may have gained a lot of flexibility. However such gain should not come at the cost of lack of compile-time optimization. People often enough turn towards an FFI when some fast library is needed they don't want to port... I believe there's a need for some translate-*-FORM protocol, which can generate code for known types, and eliminate run-time GF calls when present, by which I mean that user-defined custom fancy converters may still call GF at run-time, they are welcome. But GF is not the typical case, which gets optimized as tight as can be. Regards, Jorg Hohle

5 4

[cffi-devel] Bit based structs
by Roger Sen Montero 07 Feb '06

07 Feb '06

Hi all!! Does defcstruct support bit fields? I've been looking the docs and code but found nothing. Supose I need to support a struct like: typedef struct ilovebits { int one:1; int two:1; int three:1; int four:1; int rest:4; } lovely; What should I do? Regards, rogersm.

1 0

AW: [cffi-devel] Re: unnamed callback (closures?)
by Hoehle, Joerg-Cyril 07 Feb '06

07 Feb '06

Jan Rychter wrote: > The problem is that the C side insists on identifying > callbacks to delete by a function pointer. In this approach all > callbacks would get the same function pointer on the C side -- that > of the dispatching callback function on the Lisp side. Which means I > won't be able to delete them on the C side afterwards. Then just don't delete them. If the C side only sees only one, there won't be such a memory loss (but analyse that, maybe there's some for each new window?). OTOH, as Martin Simmons said, other GUI (which one are you using?) don't suffer from such a braindamage. And more importantly, named callbacks are supported by all Lisps (?), whereas lambdas only work with sbcl + clisp. >2. I was actually wrong when I said I knew when the callback >would be no > longer needed. Not quite sure what to do about this yet. I know next > to nothing about finalizers -- can I expect them to be supported? Forget about finalizers. *YOU* are the only one to be able to tell when the other side has finished using a resource. A good time to free resource is generally when the application exists... Regards, Jörg Höhle.

1 0