September 2005 - cffi-devel - mailman3.common-lisp.net

[cffi-devel] belated mentoring
by Kenny Tilton 13 Sep '05

13 Sep '05

Just noticed foreign-alloc (while looking for foreign-allocate <g>), which reminds me of Tilton's Law of Abbreviation: Only abbreviate names (a) longer than six characters (b) where a reasonable abbreviation saves more than half the characters. Exceptions are made where a well-known abbreviation exists, such as serial-no instead of serial-number (number having only six characters). The nice thing about a strict abbreviations rule is the elimination of coding anxiety / guesswork / aproposing. -- Kenny Why Lisp? http://wiki.alu.org/RtL_Highlight_Film "I've wrestled with reality for 35 years, Doctor, and I'm happy to state I finally won out over it." Elwood P. Dowd, "Harvey", 1950

2 1

Re: [cffi-devel] next review steps
by Kenny Tilton 13 Sep '05

13 Sep '05

Frank Goenninger - PRION Consulting wrote: > Kenny, > > here it is (hoping I am the first one to respond)... ;-) You are, thanks, now get some sleep. :) kt

1 0

[cffi-devel] Important lispworks bugfix and callbacks improvement
by Luis Oliveira 13 Sep '05

13 Sep '05

Hello, Hmm, I should setup one of those daily changelog diff generators. Tue Sep 13 06:40:15 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Changes to callbacks. - Renamed make-callback to %defcallback and shifted the responsability of storing the address in the callback-ptr property to the cffi-sys backend. Previously in allegro and lispworks the host ffi's callback definitions were non toplevel, which is probably not right. - Use gensyms instead of callback-symbol-name (removed that function from utils.lisp) Mon Sep 12 07:52:52 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Stack allocation in cffi-lispworks and tweak to run-tests - Lispworks's with-foreign-ptr stack allocates again. - run-tests now asks if we want to run the tests several times. Mon Sep 12 04:10:53 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Fixed bug in cffi-lispworks - foreign-alloc now uses type :byte. - make-callback specifies more options to define-foreign-callable. -- Luis Oliveira luismbo (@) gmail (.) com Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

1 0

[cffi-devel] Proposal for Array Type
by Rayiner Hashem 13 Sep '05

13 Sep '05

A while ago, somebody pointed out that structures with array members don't compile properly on Verrazano. In looking at the issue, I realized that arrays are something of a special case in C-FFI, as they are morally type-specifiers in defcstruct declarations, but do not have underlying types. Therefore, I propose a new C-FFI declaration: (defcarray <foreign-type-name> <base-type> <length>) eg: int[5] -> (defcarray int-array :int 5) An explicit nil for the <length> argument would be equivalent to: (define-foreign-type int-array () ':pointer) All array types would be passed as :pointer under the hood, just like structure types. Philosophically, it might be undesirable to add array types in order to keep with the minimally-typed design of C-FFI. However, arrays, from the point of view of C semantics, are much more similar to structures than to pointers. Indeed, in the C machine model, a structure is little more than a hetrogenous array. Thus, the treatment of arrays versus structures in the C-FFI API presents something of an inconsistency, as: (defcstruct foo (a1 :int) (a2 :int)) (define-foreign-type bar () 'foo) (defcstruct baz (q bar)) is perfectly legal code, while: (defcarray foo :int 2) (define-foreign-type bar () 'foo) (defcstruct baz (q bar)) is unrepresentable. This issue makes C-FFI somewhat incongruous with C, because in C, a structure field as a name and a type, with array's being incorporated into the type system, while in C-FFI, a structure field has a name, a type, and an optional length, with no explicit array type. Moreover, since arrays are not types, the usual type machinery that can be used on array types in C header files must be translated into other things in C-FFI declaration files. Now, there is a somewhat selfish undertone to this proposal, as the lack of array types in C-FFI necessitates a rather ugly special case in Verrazano's C-FFI backend. However, I believe that an array type would be an improvement for hand-made C-FFI declarations as well, specifically for cases where an array type is not specified directly in a structure, but used via typedef in a structure. Sincerely, Rayiner Hashem

2 1

[cffi-devel] verrazano-support requirement
by Kenny Tilton 12 Sep '05

12 Sep '05

Two thoughts re the verrazano-support package: minor: the openal-library generated by vzn requires verrrazano-support (it tries to load it), but the bindings per se do not. A small nicety would be to avoid an unnecessary dependency. But this goes away if... how about moving vzn support code into CFFI? Maybe we establish a rule of thumb: anything required at runtime goes in CFFI, and VZN is not needed at runtime (including any "support" package). I know vzn wants to support multiple backends, but multiple Lisp universal FFI backends? Why? UFFI is known to need an update, but an update never comes and CFFI is probably the last straw for UFFI. Hang on: is a better alternative to write the VZN-support code directly into the bindings file? Perhaps as an option the defbinding? I would just like to see xxx-library.lisp work as a standalone set of bindings, either by adding to it or by requiring certain code of any candidate backend. Thoughts? -- Kenny Why Lisp? http://wiki.alu.org/RtL_Highlight_Film "I've wrestled with reality for 35 years, Doctor, and I'm happy to state I finally won out over it." Elwood P. Dowd, "Harvey", 1950

1 0

[cffi-devel] foreign-alloc
by Luis Oliveira 12 Sep '05

12 Sep '05

Hello, Kenny complained about foreign-object-alloc not being documented a few hours ago which reminded me about the foreign-alloc vs. foreign-object-alloc issue James and I briefly discussed on IRC. I would like to experiment with a typed pointer interface (something similar to CMUCL/SBCL's Alien objects) and was planning to call those pointers "foreign objects" in which case foreign-object-alloc would be confusing since all it does is allocate raw memory. One idea James suggested was to move foreign-object-alloc's functionality to foreign-alloc and export the lower level %foreign-alloc (renamed to malloc or something similar). [Or just have the users use foreign-alloc with a :char type, why not?] Well, but why not improve (and hopefully not bloat) foreign-alloc. Here's a proposed interface based mostly on Lispworks's ffi:allocate-foreign-object and CL's make-array: foreign-alloc ============= Syntax ------ -- Function: foreign-alloc type &key initial-element initial-contents (count 1) => pointer Arguments and Values -------------------- _type_ A foreign type. _initial-element_ A Lisp object. _initial-contents_ A sequence. _count_ /Note: probably :nelems would be a better name?/ An integer bigger than or equal to 1. 1, by default. _pointer_ A foreign pointer to the newly allocated memory. Description ----------- The `foreign-alloc' function allocates enough memory to hold _count_ objects of type _type_ and returns a _pointer_. This memory must be explicitly freed using `foreign-free' once it is no longer needed. If _initial-element_ is supplied, it is used to initialize the _count_ objects the newly allocated memory holds. If an _initial-contents_ sequence is supplied, it must have a length less than or equal to _count_ and each of its elements will be used to initialize the contents of the newly allocated memory. /Note: Should the remainder be zero filled?/ /Note: also is initial-contents is supplied and count isn't then count will be (length initial-contents) [plus 1, probably]. See note in the examples./ _initial-element_ and _initial-contents_ are mutually exclusive. Examples -------- CFFI> (foreign-alloc :char) #<A Mac Pointer #x1022E0> ; A pointer to 1 byte of memory. CFFI> (foreign-alloc :char :count 20) #<A Mac Pointer #x1022E0> ; A pointer to 20 bytes of memory. CFFI> (foreign-alloc :int :initial-element 12) #<A Mac Pointer #x1022E0> CFFI> (mem-ref * :int) 12 CFFI> (foreign-alloc :int :initial-contents '(1 2 3)) #<A Mac Pointer #x1022E0> CFFI> (loop for i from 0 below 3 collect (mem-ref * :int i)) (1 2 3) CFFI> (foreign-alloc :int :initial-contents #(1 2 3)) #<A Mac Pointer #x1022E0> CFFI> (loop for i from 0 below 3 collect (mem-ref * :int i)) (1 2 3) ;; Another note: what should should be in (mem-ref ** :int 3)? ;; Zero? Ie. Should these previous two calls to foreign-alloc ;; allocate 3*sizeof(int) or 4*sizeof(int) bytes? Lispworks seems to ;; do the latter. CFFI> (foreign-alloc :string :initial-element "foo") ;; I think this should do something similar to CLISP's ;; FFI:ALLOCATE-DEEP, and it should Just Work if we apply the :to-c ;; translator, which will allocate space for "foo". ;; A problem here is freeing this memory. Should we tell the user ;; he has to free the complex stuff inside by himself. Or should we ;; come up with something similar to CLISP's FFI:FOREIGN-FREE when ;; passed ":full t". (In this case CFFI:FOREIGN-FREE would have to ;; take an optional type or something to describe what needs ;; freeing.) Comments are most welcome. -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

2 4

[cffi-devel] uffi-compat, foreign-alloc, bug fixes
by Luis Oliveira 09 Sep '05

09 Sep '05

Hello, Kenny, telling the c.l.l savages to beat me up was mean :-) so I completed what was missing in uffi-compat and it passes all tests under SBCL and CLISP. The real UFFI itself failed a couple of tests on Allegro but passes all tests using UFFI-COMPAT, ahah. Isn't that sweet? So here are the new changes in my darcs tree. Fri Sep 9 02:00:22 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * foreign-alloc and uffi-compat - removed foreign-object-alloc, added cffi:foreign-alloc with new options (including the functionality foreign-object-alloc provided). - renamed the cffi-sys:foreign-alloc's to cffi-sys:%foreign-alloc. - updated respective documentation. - improved mem-aref's documentation including new examples. - fixed with buf with lisp-string-to-foreign not handling empty strings correctly. added regression test. - fixed translate-to-c and translate-from-c (not exported yet) - fixed uffi-compat bugs, implemented all operators (including a bunch that aren't documented in UFFI's manual) except a foreign-string-length which is weird and not used anywhere. - added "UFFI" as a nickname for the cffi-uffi-compat package. Thu Sep 8 07:12:53 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Fixed Allegro warning, removed #-sbcl from callback test - Don't use call-direct on functions with 0 arguments. [Allegro] - Added a regression test for this and a test to see what void functions return. (the latter will probably fail on some ports) - SBCL has fixed the callback bug in version 0.9.4.11, removed the #-sbcl before the test that was triggering the bug. Wed Sep 7 06:31:19 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Handle empty structures gracefully - Fixed notice-foreign-structure-definition to handle empty struct definitions correctly. - Added new test for this: STRUCT.ALIGNMENT.5 -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

2 2

[cffi-devel] various
by Kenny Tilton 08 Sep '05

08 Sep '05

0. My OpenAL demo now works fine. The segmentation fault was because I was passing a Lisp rational where a float was needed. My poor-man's version of VZN generated wrappers which coerced all arguments before calling the true C binding. A quick glance at the CFFI defcfun expansion suggests automatic conversion is being done there, as well. What am I missing? Does the mechanism not attempt converting rationals to floats? 1. I think VZN should export all symbols. It was a total pain doing those manually. Besides, C does not do exported vs. imported, so trying not to export all symbols is an "extra". And the cost of that extra is the aggravation of having to manually cobble together all the symbols to be exported. 2. The latest VZN gens two warnings: ; While compiling (METHOD CHECK-AND-MARK-ARTIFICIAL (T)) in C:\0devtools\verrazano\frontend\simplifier.lisp: Warning: Variable NODE-OR-EDGE is never used. ; While compiling (METHOD GENERATE-PACKAGE ((EQL :CFFI-BACKEND) T T)) in C:\0devtools\verrazano\cffi-backend\generator.lisp: Warning: variable BK is used yet it was declared ignored 3. (CFFI) As advertised in earlier epistles, these are coming from somewhere: ; While compiling (:TOP-LEVEL-FORM "openal-library.lisp" 2333) in C:\0dev\cl-openal\openal-library.lisp: Warning: :call-direct ignored; no argument types specified. ; While compiling (:TOP-LEVEL-FORM "openal-library.lisp" 2333) in C:\0dev\cl-openal\openal-library.lisp: Warning: :call-direct ignored; no argument types specified. ; While compiling (:TOP-LEVEL-FORM "openal-library.lisp" 2333) in C:\0dev\cl-openal\openal-library.lisp: Warning: :call-direct ignored; no argument types specified. This is AllegroCL. Looks like CFFI needs to avoid inserting the :call-direct in some cases. Let me know if you need the actual code and I will try to figure out which are squawking. 4. (VZN) al.h or someone has: #define AL_FALSE 0 and #define AL_NO_ERROR AL_FALSE. The second define does not make it into the bindings because it does not look like a numeric constant. I can add it to the generated bindings manually, but then I have to do it every time I regen the bindings. Would it make sense to allow users to provide code in the defbinding form which will get blindly copied into the output?: (defbinding "openal-library" (nicknames "al") (flags "") (include "AL/al.h" "AL/alc.h" "AL/alut.h") (export "alenable" "aldisable") (export....) (override "") (append (defparameter +al-no-error+ 0))) Btw, why not defconstant when translating #defines? Anyway, congrats to all on the first "live" set of bindings. -- Kenny Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film "I've wrestled with reality for 35 years, Doctor, and I'm happy to state I finally won out over it." Elwood P. Dowd, "Harvey", 1950

3 5

[cffi-devel] OpenAL or Bust
by Kenny Tilton 07 Sep '05

07 Sep '05

My OpenAL code also uses stuff from alc.h, so I modified my binding def: (defbinding "openal-library" (nicknames "al") (flags "") (include "AL/al.h" "AL/alc.h" "AL/alut.h") (export "alenable" "aldisable") (override "")) This yielded amongst other good stuff: (cffi:defcstruct alcdevice-struct) ...on which we get divide-by-zero in CFFI because notice-foreign-struct-definition initializes max-align to zero and that never gets bumped up (because there are no slots and that is when it gets adjusted) and then gets used in a rem to determine padding. Checking alc.h and the rest of the al includes, I do not see any actual definition for alcdevice-struct. I guess the type is needed to define the functions which get passed a pointer to a device struct, but the actual make-up of the struct is pure internal. I will let you all fight it out, but it looks as if CFFI needs to be more tolerant? I will try initializing the max-align to 2 and see how much further I get. -- Kenny Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film "I've wrestled with reality for 35 years, Doctor, and I'm happy to state I finally won out over it." Elwood P. Dowd, "Harvey", 1950

3 3

Re: [cffi-devel] OpenAL or Bust
by Kenny Tilton 07 Sep '05

07 Sep '05

Rayiner Hashem wrote: > Hmm, not only is there not a definition of alcdevice-struct on my > machine, but GCC-XML's output doesn't have a "struct" node > corresponding to it. There is a typedef referring to it, which > Verrazano reproduces in the Lisp binding, but it never emits a defcstruct. > > In any case, I just inserted: > > struct my_foo {}; > > Into one of my test bindings, and it did generate: > > (cffi:defcstruct my-foo) > > CFFI on SBCL doesn't seem to have a problem with it, though. After > loading the Lisp binding, I executed: > > (cffi:defcstruct my-foo2 (data my-foo)) > > and it compiled that just fine. So my guess is that its an > Allegro-specific (or at least, non-SBCL-specific) issue. > > > In principle, CFFI should have no problem with such a declaration, ... Yeah, it just looks like a simple bug arising from no test having been done with no slots. My hack to start at 2 has got me to the point where my demo runs and plays the expected WAV file. It then gets a segmentation fault on an alSourcef call, which I will look at tomorrow. Rayiner, did you get my note about the missing in-package form? -- Kenny Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film "I've wrestled with reality for 35 years, Doctor, and I'm happy to state I finally won out over it." Elwood P. Dowd, "Harvey", 1950

1 0