December 2005 - cffi-devel - mailman3.common-lisp.net

[cffi-devel] mid-level implementation-specific patches (was: uffi and CUSTOM:* FOREIGN-ENCODING* from clisp)
by Hoehle, Joerg-Cyril 09 Dec '05

09 Dec '05

Hi, Pascal Bourguignon's proposed patch to src/strings.lisp adds #+clisp, #-clisp to the source. (cf. mailing archives from Mid-November). I noticed the wide lack of such #+implementation-specific code in the CFFI sources. I like it, but it's limited. I would like to ask what are your thoughts on integration of such code? I.e., there are a few places (e.g. in strings.lisp) where the CLISP version could clearly benefit from specific code. One of the typical improvements is to get rid of (loop ... do mem-ref #) to copy array contents elementwise and have the FFI built-ins handle them (using native loops). Remember, LOOP compiles to byte code, not tight memcpy()-like code that CMUCL can achieve. BTW, what happened to the proposed block memory interface? The general idea is that while the low-level API is fine, as it is currently split across each implementation in cffi-xyz.lisp, there's also a need for high-level implementation-specific optimizations. Remember, as Peter Norvig said, a Lisp compiler is not going to help optimizing a Prolog compiler implemented in Lisp. Prolog needs its own optimizations. Same for FFI code. Regards, Jörg Höhle.

3 2

[cffi-devel] why avoid NIL as NULL pointer in CFFI?
by Hoehle, Joerg-Cyril 09 Dec '05

09 Dec '05

Hi, It's the first time I come across CFFI after James Bielmann's initial questions on CFFI design in June 2005. I had a short glance at the CLISP implementation and it immediately struck me that CFFI dislikes CLISP's consistent use of NIL for a NULL FFI pointer. Why? I find using separate types for NULL and regular pointers extremely beneficial to programs. The "NIL is not a foreign-address" allows for cheap safety nets, i.e. my original AFFI (FFI for Amiga-CLISP, design in 1994/95) would error out when trying to mem-read from a NIL "pointer". The application won't crash. OTOH, I can understand that native-code compilers prefer homogenous types be used throughout the code (so i.e. type declarations look simpler, and maybe compilers like CMUCL's Python can avoid some boxing), but CLISP has been bytecode for over 13 years, which allows for different designs. CLISP is very consistent about accepting and returning NIL for NULL pointers, so why does CFFI go to some length to reintroduce foreign pointers whose address is 0 instead of just using NIL? BTW, IIRC my own UFFI-wrapper macros for CLISP, there was no problem at all with defining (deftype ffi-kind (or NULL FFI-ADDRSS FFI-VARIABLE FFI-FUNCTION)). So CLOS slots could have valid :type declarations. FWIW, it's Bruno Haible whom whoever (James?) wrote a comment about beer in cffi-clisp.lisp owes a beer for exporting the interface to alignment of foreign types. It's been there since 1995. Regards, Jörg Höhle.

2 1

[cffi-devel] Re: def-foreign-type and with-foreign-object
by Hoehle, Joerg-Cyril 09 Dec '05

09 Dec '05

Hi, Luis Oliveira wrote: This is clearly a bug in this clsql code. That should be (deref-pointer phdbc 'sql-handle), note the missing quote. Yaroslav, if you look at my UFFI-CLISP wrapper macros, you'll find an outcommented definition of def-foreign-function which is useful to compile and type-check UFFI types, without actually calling FFI functions. UFFI will widly accept quoted and unquoted types in many places, yet the documentation says otherwise. It's not a feature, it's a bug. Not knowing what gets evaluated and what not is a pain in macros. With that help, I checked all of clsql months ago (even the backends that I cannot run), and I thought I had all errors reported to Keven. Well, maybe not. Regards, Jörg Höhle

1 0

[cffi-devel] New (backwards incompatible) changes in cffi-luis.
by Luís Oliveira 09 Dec '05

09 Dec '05

Hello, There have been some backwards incompatible changes: 1. defcunion and defcstruct's slot definition now expect &key count instead of &optional count. 2. s/ptr/pointer/ on the cffi functions names (see the changelog changelog for the complete list). 3. foreign-slot-address was also renamed to foreign-slot-pointer. Also, there are two new functions: CFFI:MAKE-POINTER and CFFI:POINTER- ADDRESS. Changelog follows. Fri Dec 9 02:28:21 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Lots of renaming and 2 new functions - Renamed: null-ptr -> null-pointer, null-ptr-p -> null-pointer, inc-ptr -> inc-pointer, with-foreign-ptr -> with-foreign-pointer, with-foreign-ptr-as-string -> with-foreign-pointer-as-string, foreign-slot-address -> foreign-slot-pointer, get-var-ptr -> get-var-pointer and foreign-symbol-ptr -> foreign-symbol-pointer. - Updated various references to the old names in the cffi code itself, examples, tests and the manual. - New functions: cffi(-sys):pointer-address and cffi(-sys):make- pointer. - Use these in uffi-compat. - Documented them in the manual. - New tests: POINTER.1 and POINTER.2 Thu Dec 8 06:58:07 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Remove objects.lisp (new file: late-types.lisp) Thu Dec 8 06:34:12 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * defcunion :count option for slots - Make the defcunion slots' count option be a keyword like defcstruct. Fix the related docs and tests. Mon Dec 5 01:46:28 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Apply translations for simple slots. - Make the foreign-slot-value method on simple-struct-slot apply the adequate type translator. Same thing for the setter. - Test this in STRUCT.STRING.1 and STRUCT.STRING.2 Sat Dec 3 03:48:26 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Missing changes in last patch... Sat Dec 3 01:12:23 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * Documention changes - commented out some example structures that use unimplemented features. - documented the recent DEFCSTRUCT changes; added related example. - mention that lisps that don't support FOREIGN-FUNCALL support DEFCFUN anyway. - new example with FOREIGN-FUNCALL - fix the comments in types.lisp about ABIs. - (non doc related) make cffi-tests :use REGRESSION-TEST instead of RT. -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www.iro.umontreal.ca/translation/registry.cgi?team=pt

1 0

[cffi-devel] problem with cffi-luis-051205-0148 ??
by Rick Taube 07 Dec '05

07 Dec '05

Im having a problem loading file using cffi-luis-051205-0148/ (the file works with cffi-luis-051114-1956. ) the error happens when i eval my first defcstruct: (cffi:defcstruct tmidi-sex-1 (link midi-sexptr) (data byte 11)) Here is a trace in openmcl 1.0/darwin. the problem also occurs in linux sbcl 0.9.7 and cmucl 19c: ? (cffi:defcstruct tmidi-sex-1 (link midi-sexptr) (data byte 11)) ? > Error in process listener(1): Odd length keyword list: (11) > While executing: CCL::CHECK-KEYWORDS > Type :POP to abort. Type :? for other options. 1 > (:b t) (F0135BE0) : 0 "CCL::CHECK-KEYWORDS" 308 0 : 4 ("saved SAVE0") 1 : (11) ("saved SAVE1") 2 : NIL ("saved SAVE2") 3 : (DATA BYTE 11) ("saved SAVE3") 4 : NIL ("saved SAVE4") 5 : 4 ("saved SAVE5") (F0135BF0) : 1 "CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION" 348 0 CFFI::NAME-AND-OPTIONS: TMIDI-SEX-1 ("required") 1 CFFI::SLOTS: ((LINK MIDI-SEXPTR) (DATA BYTE 11)) ("required") 2 : (EVAL-WHEN (:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE) (CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION 'TMIDI-SEX-1 '((LINK MIDI-SEXPTR) (DATA BYTE 11)))) ("saved SAVE0") 3 : (:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE) ("saved SAVE1") 4 : NIL ("saved SAVE2") 5 : EVAL-WHEN ("saved SAVE3") 6 : #<CCL::DESTRUCTURE-STATE #x294B76> ("saved SAVE4") 7 : ((:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE) (CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION 'TMIDI-SEX-1 '((LINK MIDI-SEXPTR) (DATA BYTE 11)))) ("saved SAVE5") 8 : ((CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION 'TMIDI-SEX-1 '((LINK MIDI-SEXPTR) (DATA BYTE 11)))) ("saved SAVE6") 9 : MACROEXPAND-1 ("saved SAVE7") 10 #:WHOLE495: (TMIDI-SEX-1) 11 #:ARGS496: #<CCL::DESTRUCTURE-STATE #x294AF6> 12 CFFI::NAME: TMIDI-SEX-1 13 #:KEYS497: NIL 14 CFFI::SIZE: NIL 15 CFFI::STRUCT: #<FOREIGN-STRUCT-TYPE TMIDI-SEX-1> 16 CFFI::SLOTNAME: DATA (F0135C00) : 2 NIL NIL (F0135C10) : 3 "CCL::CALL-CHECK-REGS" 72 0 CCL::FN: CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION ("required") 1 CCL::ARGS: (TMIDI-SEX-1 ((LINK MIDI-SEXPTR) (DATA BYTE 11))) ("rest") 2 CCL::OLD-REGS: ((EVAL-WHEN . #2=(#1=(:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE) . #3=((CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION 'TMIDI-SEX-1 '((LINK MIDI-SEXPTR) (DATA BYTE 11)))))) #1# NIL EVAL-WHEN #<CCL::DESTRUCTURE-STATE #x294B76> #2# #3# MACROEXPAND-1) 3 : CFFI::NOTICE-FOREIGN-STRUCT-DEFINITION (F0135C20) : 4 NIL NIL (F0135C30) : 5 "CCL::CHEAP-EVAL-IN-ENVIRONMENT" 1536 0 : #<SVAR *PACKAGE* 43 #x802740E> ("saved SAVE0") 1 : #<SYNONYM-STREAM to *TERMINAL-IO* #x838935E> ("saved SAVE1") 2 : (NIL) ("saved SAVE2") 3 : (BYTE) ("saved SAVE3") 4 : 0 ("saved SAVE4") 5 : T ("saved SAVE5") 6 : #<SYNONYM-STREAM to *TERMINAL-IO* #x8389296> ("saved SAVE6") 7 : #<Anonymous Function #x80E8E4E> ("saved SAVE7") (F0135C40) : 6 NIL NIL (F0135C50) : 7 "CCL::TOPLEVEL-EVAL" 152 0 CCL::FORM: (CFFI:DEFCSTRUCT TMIDI-SEX-1 (LINK MIDI-SEXPTR) (DATA BYTE 11)) ("required") 1 : NIL ("optional") 2 : (CFFI:DEFCSTRUCT TMIDI-SEX-1 (LINK MIDI-SEXPTR) (DATA BYTE 11)) ("saved SAVE0") 3 CCL:*LOADING-FILE-SOURCE-FILE*: NIL (:SAVED-SPECIAL) 4 PACKAGE: #<Package "MIDISHARE"> (F0135C60) : 8 "CCL::READ-LOOP" 848 0 : 0 ("keyword") 1 : T ("key-supplied-p") 2 : NIL ("keyword") 3 : NIL ("key-supplied-p") 4 : #<SYNONYM-STREAM to *TERMINAL-IO* #x838935E> ("keyword") 5 : T ("key-supplied-p") 6 : #<SYNONYM-STREAM to *TERMINAL-IO* #x8389296> ("keyword") 7 : T ("key-supplied-p") 8 : #S(CCL::SHARED-RESOURCE :NAME "Shared Terminal Input" :LOCK #<RECURSIVE-LOCK [ptr @ #x301880] #x8389206> :PRIMARY-OWNER #<PROCESS listener(1) [Active] #x8388F66> :PRIMARY-OWNER-NOTIFY #<CCL:SEMAPHORE #x83891D6> :CURRENT-OWNER NIL :REQUESTORS #<DLL-HEADER #x83891C6>) ("saved SAVE0") 9 : #<SVAR CCL::*AUTO-FLUSH-STREAMS* 366 #x811DA5E> ("saved SAVE1") 10 : #<A Mac Pointer #x300A80> ("saved SAVE2") 11 : 0 ("saved SAVE3") 12 : 0 ("saved SAVE4") 13 : 0 ("saved SAVE5") 14 : 0 ("saved SAVE6") 15 : 0 ("saved SAVE7") 16 CCL::*BREAK-LEVEL*: 0 (:SAVED-SPECIAL) 17 CCL::*LAST-BREAK-LEVEL*: 0 (:SAVED-SPECIAL) 18 CCL:*LOADING-FILE-SOURCE-FILE*: "ccl:l1;l1-boot-3.lisp.newest" (:SAVED-SPECIAL) 19 CCL::*IN-READ-LOOP*: NIL (:SAVED-SPECIAL) 20 ***: NIL (:SAVED-SPECIAL) 21 **: NIL (:SAVED-SPECIAL) 22 *: NIL (:SAVED-SPECIAL) 23 +++: NIL (:SAVED-SPECIAL) 24 ++: NIL (:SAVED-SPECIAL) 25 +: NIL (:SAVED-SPECIAL) 26 ///: NIL (:SAVED-SPECIAL) 27 //: NIL (:SAVED-SPECIAL) 28 /: NIL (:SAVED-SPECIAL) 29 -: NIL (:SAVED-SPECIAL) 30 #:ABORT-BREAK: #<RESTART CCL:ABORT-BREAK #x294CBE> 31 #:ABORT: #<RESTART ABORT #x294C96> 32 #:G24872: (#<RESTART ABORT #x294C96> #<RESTART CCL:ABORT-BREAK #x294CBE>) 33 CCL::%RESTARTS%: ((#<RESTART CCL:ABORT-BREAK #x294DF6> #<RESTART ABORT #x294E1E>)) (:SAVED-SPECIAL) 34 : (CFFI:DEFCSTRUCT TMIDI-SEX-1 (LINK MIDI-SEXPTR) (DATA BYTE 11)) 35 CCL::PATH: NIL 36 : T (F0135CA0) : 9 "CCL:TOPLEVEL-LOOP" 88 (F0135CC0) : 10 "Anonymous Function #x80D2DBE" 68 0 CCL::INIT-FILE: "home:openmcl-init" (:INHERITED) (F0135CD0) : 11 NIL NIL (F0135CE0) : 12 "Anonymous Function #x811DA6E" 728 0 CCL::CLOSE-STREAMS: NIL (:INHERITED) 1 CCL::CLEANUP-FUNCTION: #<CCL:COMPILED-LEXICAL-CLOSURE #x838912E> (:INHERITED) 2 CCL::INITIAL-FUNCTION: #<CCL:COMPILED-LEXICAL-CLOSURE #x8389116> (:INHERITED) 3 CCL:OUTPUT-STREAM: #<FD-CHARACTER-OUTPUT-STREAM (TTY/1) #x83896EE> (:INHERITED) 4 CCL:INPUT-STREAM: #<FD-CHARACTER-INPUT-STREAM (TTY/0) #x8389966> (:INHERITED) 5 : (#<RESTART CCL:ABORT-BREAK #x294DF6> #<RESTART ABORT #x294E1E>) ("saved SAVE0") 6 : #<PROCESS listener(1) [Active] #x8388F66> ("saved SAVE1") 7 : #<RESTART CCL:ABORT-BREAK #x294DF6> ("saved SAVE2") 8 *TERMINAL-IO*: #<ECHOING-TWO-WAY-STREAM input #<FD-CHARACTER-INPUT-STREAM (TTY/0) #x8389966>, output #<FD-CHARACTER-OUTPUT-STREAM (TTY/1) #x83896EE> #x8389436> (:SAVED-SPECIAL) (F0135D00) : 13 "CCL::RUN-PROCESS-INITIAL-FORM" 400 0 CCL::INITIAL-FORM: (#<CCL:COMPILED-LEXICAL-CLOSURE #x8388E86>) ("required") 1 : #<PROCESS listener(1) [Active] #x8388F66> ("saved SAVE0") 2 : 0 ("saved SAVE1") 3 : 0 ("saved SAVE2") 4 CCL::EXITED: NIL 5 #:G51: (CCL:PROCESS-RESET) 6 CCL::%HANDLERS%: NIL (:SAVED-SPECIAL) 7 #:ABORT: #<RESTART ABORT #x294E1E> 8 CCL::%RESTARTS%: NIL (:SAVED-SPECIAL) 9 : #<CCL:COMPILED-LEXICAL-CLOSURE #x8388E86> (F0135D30) : 14 NIL NIL (F0135D40) : 15 "Anonymous Function #x80DB61E" 152 0 CCL::INITIAL-FORM: (#<CCL:COMPILED-LEXICAL-CLOSURE #x8388E86>) ("required") 1 : 0 ("saved SAVE0") 2 CCL:*CURRENT-PROCESS*: #<PROCESS Initial(0) [Active] #x80DC106> (:SAVED-SPECIAL) 3 CCL::SYMS: (CCL::*FREE-XPS* CCL::*FREE-CIRCULARITY-HASH-TABLES* CCL::*OPERAND-VECTOR-FREELIST* CCL::*LAP-INSTRUCTION-FREELIST* CCL::*LAP-LABEL-FREELIST* CCL::*VINSN-VARPARTS* CCL::*VINSN-LABEL-FREELIST* CCL::*VINSN-FREELIST* CCL::*LCELL-FREELIST* CCL::*LREG-FREELIST* CCL::%TOKEN-STRINGS% *READTABLE* CCL::*BACKTRACE-CONTEXTS* CCL::*FAKE-STACK-FRAMES* CCL::*CONS-POOL* *PACKAGE*) 4 VALUES: (NIL NIL #<POOL #x838BD16> #<POOL #x838BCFE> #<POOL #x838BCE6> #<POOL #x838BCCE> #<POOL #x838BCB6> #<POOL #x838BC9E> #<POOL #x838BC86> #<POOL #x838BC6E> #<POOL #x838BC56> #<READTABLE #x80E1216> NIL NIL #<POOL #x838BC0E> #<Package "COMMON-LISP-USER">) (F0135D60) : 16 "Anonymous Function #x80CE806" 172 0 CCL::TCR: 788032 (:INHERITED) 1 CCL::THREAD: #<LISP-THREAD listener [tcr @ #x301900] #x838906E> (:INHERITED) 2 CCL::*CURRENT-LISP-THREAD*: #<LISP-THREAD Initial [tcr @ #x300430] #x80CEAB6> (:SAVED-SPECIAL) 3 CCL::INITIAL-FUNCTION: (#<Anonymous Function #x80DB61E> #<PROCESS listener(1) [Active] #x8388F66> (#<CCL:COMPILED-LEXICAL-CLOSURE #x8388E86>)) 4 : #<Anonymous Function #x80DB61E> 1 >

2 2

[cffi-devel] Recent changes: makefiles, defcstruct, foreign-string-alloc/with-foreign-string
by Luís Oliveira 02 Dec '05

02 Dec '05

Hello, Changelog with the recent changes in cffi-luis follows: Fri Dec 2 07:56:37 WET 2005 Luis Oliveira <loliveira(a)common-lisp.net> * makefiles, defcstruct, foreign-string-alloc/with-foreign-string - Use variables for the lisps' paths so that one can easily pick different versions to run the tests with. eg: CMUCL=/path/to/lisp make test-cmucl - FOREIGN-STRING-ALLOC and WITH-FOREIGN-STRING: check if what we're passing as a string is actually a string. This avoids (foreign-string-alloc nil) => <pointer to empty C string> - DEFCSTRUCT now takes an optional :size option and each slot takes an optional :offset parameter that overrides the normal offset. This enables partial definitions of C structs. - The previous change forces to make the COUNT option for the slots to be a keyword option. Update the examples accordingly. ^^ [ugh, beautiful wording!] - tests/Makefile: support for SunOS. Tue Nov 15 02:27:28 WET 2005 James Bielman <jamesjb(a)jamesjb.com> * add and export WITH-FOREIGN-STRINGS to uffi-compat -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www.iro.umontreal.ca/translation/registry.cgi?team=pt

1 0