- cffi-devel - mailman3.common-lisp.net

[cffi-devel] Struct alignment/size bug fixed, win32 tweaks, etc..
by Luis Oliveira 02 Sep '05

02 Sep '05

Hello, About the struct alignment issue, I found some docs here: http://refspecs.freestandards.org/ and fixed the bug following the instructions in "System V Application Binary Interface Intel386 Architecture Processor Supplment, Fourth Edition". According to the other ABI docs under "Processor Specific ELF documents" the same rules apply at least for x86-64, MIPS and PowerPC. These documents mention System V, so I suppose it could be different for other systems. I have tested this under Win32 and MacOS X (as well as Linux) and they seem to use the same rules. So anyway, these are the new changes in cffi-luis: Fri Sep 2 08:57:50 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * CFFI User Manual update. Fri Sep 2 05:40:50 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Fix struct bug, defcvar bug and tweaked bindings.lisp - defcvar now works (more) correctly with aggregate types. - struct types now carry alignment information which is now correctly calculated (for unions too) according to the x86 ABI (tested on darwin/ppc too, but should study other ABI docs at some point). - also tail padding is now correctly added to structs, when necessary. (also according to the x86 ABI). - 4 new tests for this struct bug (reported by Wilco Greven). - tweaked tests/bindings.lisp to correctly load the library on Lispworks and to make it more portable across win/nix/etc.. Thu Sep 1 08:36:04 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Slight change to defcenum to make Corman Lisp happier. Wed Aug 31 07:46:45 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Well, lets support MINGW too. Tweaked the Makefile a bit to support MINGW too. Wed Aug 31 07:30:13 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Win32 tweaks to the test library... ... in order to work with the MSVC++ free toolkit. - correctly load the test library. - and __declspec(dllexport) to the test library functions - implement a strdup, used in 2 tests - added a compile.bat, the free toolkit doesn't seem to have nmake. Also, I don't know how to make ASDF automatically compile the lib under win32 like I did for *nix. -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

1 0

[cffi-devel] Enumerations cannot contain duplicate values
by Rayiner Hashem 02 Sep '05

02 Sep '05

Hello! I've encountered some behavior in C-FFI that I would like to discuss. When you try to define an enumeration containing duplicate values, C-FFI gives the error: "A foreign enum cannot contain duplicate values: 0." However, C enumerations can contain duplicate values, and this feature is used by real code. FLTK, for example, uses it to provide for source compatibility when the name of enumeration elements are changed. Without direct C-FFI support for this feature, the resultant C-FFI bindings do not translate well to what the user expects (enumeration keys available in the C header are not available in the Lisp binding). Verrazano works around this issue right now (by discarding duplicate keys), but it's a sub-optimal solution, since there is no way to programatically determine a consistent set of keys to discard. For example, FLTK contains an enumeration in which FL_ALIGN_TOP_LEFT and FL_ALIGN_LEFT_TOP alias each other, and FL_ALIGN_TOP_RIGHT FL_ALIGN_RIGHT_TOP alias each other. Depending on the ordering in the header file, the enumeration may end up containing an inconsistent set of names (eg: FL_ALIGN_TOP_LEFT and FL_ALIGN_RIGHT_TOP). Sincerely, Rayiner Hashem

2 1

[cffi-devel] foreign-type-size of struct types
by Wilco Greven 02 Sep '05

02 Sep '05

Hello, I'm using cffi for trying to revive the Qt bindings. Doing so I found out that for the calculation of the size of struct types, the alignment of the struct itself is not taking into account. For the following struct for example (defcstruct smoke-class (class-name string) (parents index) (class-fn :pointer) (enum-fn :pointer) (flags :unsigned-short)) cffi calculates a size of 18, which should be 20 (please correct me if I'm wrong). Attached is a patch which defines an extra method for foreign-type-size for a foreign-struct-type argument. This method is used to apply adjust-for-alignment on the newly defined struct. Greetings, Wilco

3 5

[cffi-devel] Calling functions through pointers
by Luis Oliveira 31 Aug '05

31 Aug '05

Hello, New patch in cffi-luis. Wed Aug 31 03:23:58 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * %foreign-funcall-ptr and small fixes - New CFFI-SYS operator: %foreign-funcall-ptr. Implemented for Allegro, CLISP, CMUCL, SBCL and OpenMCL. - foreign-funcall can now take either a string or a pointer as its first argument. It'll decide at macroexpansion time whether to call %foreign-funcall or %foreign-funcall-ptr. - Updated foreign-funcall's documentation. - Allegro tweaks: fix the lisp-type specification of (* :void) pointers and :unsigned-char's. In the future, it might be better to simply not specify anything were not necessary, though. - Fixed silly bug in foreign-vars.lisp. Rayiner, you can now use foreign-funcall for your C++ support. Although for maximum portability we'll have to implement a declarative interface to support Lispworks (and probably Corman and GCL). -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

1 0

[cffi-devel] Tweaks, bug fixes and documentation
by Luis Oliveira 30 Aug '05

30 Aug '05

Hello, I have updated the webpage: http://common-lisp.net/project/cffi and uploaded the documentation I did so far. Also, here's what's new in my darcs tree: Tue Aug 30 03:59:06 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Documentation, bug fixes and tweaks Bug fixes / tweaks: - fixed typo in foreign-enum-value. - defcenum: renamed argument, allow a docstring. - defcvar and defcfun: also accept symbols as names. - defcvar: added (declare (ignore value)) when read-only is true. - foreign-funcall: fixed bug; premature canonicalization of the return-type. added regression test for this. - defcfun and defcallback: use the make-gensym-list utility - defcallback: return the callback name; - strings.lisp: fixed the docstrings for the :string translators; - mem-aref: fixed bug/typo in its setf-expander; added regression test for this. - defcstruct and defcunion: allow a docstring. - new tests: callbacks.qsort and funcall.string.3 Documentation: - New file doc/Makefile for generating docs and uploading them to c-l.net. - Added every cffi function/macro that is currently exported. documented half of them or so. - colorize-lisp-examples.lisp: little script to colorize the examples in the texinfo-generated html docs. - gendocs.sh: generate the docs in various formats as well as an index page from gendocs_template. - further tweaks to doc/style.css. Sat Aug 27 09:09:00 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * More oops. - clisp: foreign-address-null is deprecated. - types: fixed bug, regarding expanders + anonymous types. - added documentation to the :boolean type Sat Aug 27 04:01:27 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Oops - Forgot to add new file objects.lisp in last patch. - Bogus type got sent by mistake in early-types.lisp - Make DEFINE-FOREIGN-TYPE return the type name. -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

1 0

[cffi-devel] Changes to the type system, among others.
by Luis Oliveira 27 Aug '05

27 Aug '05

Hello, Here's the Changelog for the new changes to my branch since my last update: Fri Aug 26 15:47:08 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Type system improvements and more. - New tests: tests/enums.lisp (4 tests) and tests/misc-types.lisp (3 tests). - New file: objects.lisp, skeleton of the safe typed pointer interface. I also added a :boolean type here, it's looking for a better home still. - (early-types.lisp): type parsers, DEFINE-TYPE-SPEC-PARSER and PARSE-TYPE. Added abbility to have anonymous types (ie. a name is no longer required for FOREIGN-TYPE). Moved DEFCTYPE to types.lisp. - (enum.lisp): improved DEFCENUM, the values for each keyword are now optional, like in C. Added the anonymous type (:enum ...) - (types.lisp): - Renamed :TO-C-ARG to :TO-C-DYNAMIC. - Renamed FOREIGN-AREF to MEM-AREF and implemented compiler macros for it. - New exported macro: DEFINE-FOREIGN-TYPE for parameterized types. - Change DEFINE-TYPE-TRANSLATOR back to the previous interface, translators take aow and we have translators available at runtime now too. - FOREIGN-TYPEDEF's now inherit their parent type's translators. - New :wrapper type, for anonymous typedefs with translators. - (strings.lisp): Renamed string to :string. Added new type, :string+ptr. - (utils.lisp): renamed let-if to bif. - (tests/callbacks.lisp): two tests were not being run for SBCL because I thought it triggered a bug but it's actually only one that triggers the bug. Fixed that. - Updated examples and tests to use :string instead of string. Sun Aug 21 14:35:13 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * A couple of minor changes. - Move #-cffi/no-foreign-funcall from cffi-tests.asd to tests/funcall.lisp and have the lisps that don't support this push the feature in cffi-<lisp>.lisp instead of defpackage. - New file: utils.lisp - Make Allegro, CMUCL, Corman and OpenMCL use callback-symbol-name (from cffi-utils) to safely intern callback symbol. - Clisp: consider (pointerp NIL) => T Thu Aug 11 05:43:53 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Clisp: use FOREIGN-ADDRESS-NULL in NULL-PTR-P Wed Aug 10 02:00:07 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Lispworks: use :byte for cffi's :char types -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

2 2

[cffi-devel] Callback Symbol Names
by James Bielman 21 Aug '05

21 Aug '05

Luis, In preparation for pulling in your changes to the main Darcs tree, I've been looking at the callback support in the cffi-luis branch. As it stands, I think CFFI-SYS::MAKE-CALLBACK does not safely intern the symbols it uses for callback names (on all the implementations but SBCL it seems). For example, if you use DEFCALLBACK from the REPL, the callback symbol is interned into the current package; in my test case, it was interned into CL-USER. Also, callbacks with the same symbol name but from different packages will redefine each other. So, I propose a separate package for callback symbols, and making sure we also use the package name of the symbol we are defining a callback for. Also, thinking back to our earlier conversation about making some sort of CFFI-UTILS package, I would suggest putting it first in the system definition so the backends can make use of it as well, then we can put CALLBACK-SYMBOL-NAME there as well. Here's a patch against cffi-luis for OpenMCL: --- old-cffi-luis/src/cffi-openmcl.lisp 2005-08-08 16:47:20.000000000 -0700 +++ new-cffi-luis/src/cffi-openmcl.lisp 2005-08-21 01:27:06.000000000 -0700 @@ -47,6 +47,8 @@ #:foreign-var-ptr #:make-callback)) +(defpackage #:cffi-callbacks) + (in-package #:cffi-sys) ;;;# Allocation @@ -248,8 +250,14 @@ ;;;# Callbacks +(defun callback-symbol-name (name) + "Return the symbol to use for the callback for NAME." + (intern (format nil "~A.~A" + (package-name (symbol-package name)) + (symbol-name name)) :cffi-callbacks)) + (defmacro make-callback (name rettype arg-names arg-types body-form) - (let ((cb-sym (intern (format nil "%callback/~A" name)))) + (let ((cb-sym (callback-symbol-name name))) `(symbol-value (defcallback ,cb-sym (,@(mapcan (lambda (sym type) (list (convert-foreign-type type) sym)) James

2 2

[cffi-devel] New patches in cffi-luis: 5/Aug
by Luis Oliveira 10 Aug '05

10 Aug '05

Hello, Yesterday was find-bug-in-lisp-implementations day, heh. Here are the new patches in my cffi branch[1]: Fri Aug 5 20:38:44 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Minor edits and changed SQRTF test. - Commented out some unused forms in src/functions.lisp. - Defined our own SQRTF in libtest.c since it's not available on win32. Fri Aug 5 20:36:58 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * SBCL: use SB-SYS:FOREIGN-SYMBOL-SAP - SBCL 0.9.3 renamed SB-SYS:FOREIGN-SYMBOL-ADDRESS to SB-SYS:FOREIGN-SYMBOL-SAP. Use this instead. Fri Aug 5 20:34:17 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Small change to DEFCFUN-HELPER-FORMS. - Make it take an extra argument "lisp-name". Hopefully I got this stuff right this time, I'm not sure yet. Fri Aug 5 20:26:31 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Make DEFCVAR use the type translation mechanism. - Also added new tests. Note, Lispworks on x86 starts behaving erratically after runing the FOREIGN-GLOBALS.SET.STRING test and eventually dies an horrible death. Also, Allegro doesn't like its memory being free()d by C. doc/allegro-internals.txt has some info on this. Should be easy to figure out. Fri Aug 5 20:19:02 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Lispworks bug fixes. - FLI:INCF-POINTER didn't increment the pointers at all. Implemented CFFI-SYS:INC-PTR using FLI:MAKE-POINTER and FLI:POINTER-ADDRESS instead. - FLI:WITH-DYNAMIC-FOREIGN-OBJECTS was causing troubles on all platforms, heap allocate for the time being. - Missing (convert-foreign-type rettype) when defining a foreign function. - FOREIGN-VAR-PTR was trying to access the var too soon, simplified it a bit too. Fri Aug 5 20:14:54 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Allegro port update, bug fixes. - Convert chars to signed-byte instead of unsigned-byte (need to figure out the issues between signed vs. unsigned char, btw) - WITH-FOREIGN-PTR was evaluating SIZE twice. Fri Aug 5 19:59:15 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * New callback interface. - New interface: DEFCALLBACK, CALLBACK, GET-CALLBACK - New CALLBACK.* tests. - Implementations of CFFI-SYS:MAKE-CALLBACK: - Allegro, passes all tests on linux/x86 and win32 fails CALLBACKS.CHAR/UNSIGNED-CHAR/SHORT/UNSIGNED-SHORT on darwin/ppc, looks like an Allegro bug. - clisp, passes all tests. - cmucl, passes all tests on linux/x86, on darwin/ppc fails CALLBACKS.CHAR/UNSIGNED-CHAR/SHORT/UNSIGNED-SHORT, looks like a bug in cmucl/ppc too. - sbcl, x86 only, passes all tests except its callbacks can't handle sap's yet, that'd be bug in sbcl too. - openmcl, fails CALLBACKS.FLOAT/DOUBLE, I reported this bug in openmcl, and it has been confirmed. - Lispworks, passes all tests on darwin/ppc, on linux/x86 and win32 there are issues unrelated to callbacks. Fri Aug 5 19:45:09 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * New type translation interface. - Removed define-type-translator, added define-type-translation with a new interface. - Make with-object-translated, translate-objects and the string type use the new interface. Fri Aug 5 19:36:31 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Minor simplification to test-op in cffi-tests.asd Thu Aug 4 05:38:27 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * :cffi/no-foreign-funcall and unexport foreign-var-ptr * Push :cffi/no-foreign-funcall to *features* when the host lisp does not support it. Btw, Frank, if you feel like testing if Allegro 7.0 on Mac has these callbacks fixed, please go ahead. :-) [1] - http://common-lisp.net/project/cffi/darcs/cffi-luis/ -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

1 1

[cffi-devel] Corman Lisp and GCL ports, patches
by Luis Oliveira 06 Aug '05

06 Aug '05

Hello, Sat Aug 6 05:11:19 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Broken "support" for GCL. - Basic pointer operations and memory access is working. - Calling foreign functions too, probably, untested. Sat Aug 6 05:00:43 WEST 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Initial support for Corman Lisp. - A few bugs and issues left, fails 5 tests. (the tests won't run automatically because Corman Lisp binds *load-truename* to NIL, so the code in tests/bindings.lisp doesn't find the C test library). I stopped implementing CFFI-SYS for GCL because I couldn't easily test it. GCL can't compile the rest of CFFI so this port dropped a few places in my priority list. I hear they're improving ANSI CL support actively so I suppose the situation will improve soon. -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

1 0

[cffi-devel] Type translators
by Luis Oliveira 05 Aug '05

05 Aug '05

Hello, I'm reworking the type translator interface. The idea is to simplify and allow for more flexibility. I'll start with some examples: This is how the boolean type would be defined with the current interface: (defctype boolean :int) (define-type-translator boolean :in (arg result-var) "Type translator to convert t/nil to a C boolean." (values `(if ,arg 1 0) nil)) (define-type-translator boolean :result (arg result-var) "Type translator to convert C booleans to t/nil." (values `(if (zerop ,arg) nil t))) First thing I'm changing is :in/:return, the names don't make sense when we think about callbacks, the :return translation is applied for the arguments and :in for the callback's return value. So I picked :to-c and :from-c. Can someone suggest better names? And this is how my interface would look like: (define-type-translation (boolean :int) "Converting between C ints and Lisp booleans." :to-c (if &value 1 0) :from-c (if (zerop &value) nil t)) It takes care of the defctype unless the first arg isn't a list. If it were just 'boolean' defctype woudn't be used (eg. when defining a type-translating for a struct type). Hmm, is this extra syntax re defctype overkill? Also it sets up &value with symbol-macrolet. Another example, the string type. Before: (defctype string :pointer) (define-type-translator string :to-c (arg result-var) "Type translator for converting Lisp strings to C string." (values `(foreign-string-alloc ,arg) `((foreign-string-free ,result-var)))) (define-type-translator string :from-c (arg result-var) "Type translator for C string values." `(foreign-string-to-lisp ,arg)) After: (define-type-translation (string :pointer) "Converting between C and Lisp strings." :in (let ((&var (foreign-string-alloc &value))) (unwind-protect (&body) (foreign-string-free &var))) :to-c (foreign-string-alloc &value) :from-c (foreign-string-to-lisp &value)) Here we have an extra translation :in, used when we are actually passing something to a foreign function (the :to-c translation is used, in case :in is undefined). Here we have to explicitly write the unwind-protect, I think the translation becomes more readable this way. :in also sets up a few more (symbol-)macros, &var and &body whose meaning should be obvious. Getting to decide where &body goes (ie. not having to be restricted to a unwind-protect form as before) gives us the ability to do this: (define-type-translation (string :pointer) "Converting between C and Lisp strings." :in (with-foreign-string (&var &value) (&body)) :to-c (foreign-string-alloc &value) :from-c (foreign-string-to-lisp &value)) This way the string is stack allocated (if the lisp supports this). In this last example, I indented the code in a different way, I'm not yet happy with the syntax, any comments or suggestions regarding syntax, naming, etc.. will be most appreciated. Also, the :to-c/:from-c translations, unlike :in, don't have access to &body as I couldn't think of a situation where this would be necessary. Am I missing something? -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www2.iro.umontreal.ca/~pinard/po/registry.cgi?team=pt

3 6