cffi-devel December 2005

cffi-devel@common-lisp.net

17 participants
66 discussions

[cffi-devel] expressive types (was: cffi/allegro crash)
by Hoehle, Joerg-Cyril 20 Dec '05

20 Dec '05

Luis Oliveira >Let me see if I got this straight, your proposal is that the :pointer >type may or may not accept NIL (but it should documented as >non-portable), No. :pointer is documented to only accept valid CFFI pointers. These are documented as opaque. Code using NIL is in error, as well as code using 0. > and that we should provide a second pointer type that >is guaranteed to translate between the null pointer and NIL. Yes, because such a type is useful. Well, for Allegro not really -- forget it. Another useful addition is a pointer type which does not accept a null-pointer, but signals an error instead, or just specifies that "passing (null-pointer) is an error". This type can be used in function declarations. The effect is two-fold: a) just from seeing the declaration, you know that NULL is not allowed. b) optionally, the transformers could check for this at run-time. It's like the difference between C-PTR and C-PTR-NULL in CLISP. Expressive type definitions are what I'd like to arrive at. There are plenty of functions in the C lib where NULL is not allowed, yet you cannot tell it from reading the .h files. The difference between NULL not allowed vs. allowed is fundamental to program verification. That's the first thing that other languages, which aim at formal verification distinguish. E.g. Cyclone is IIRC such a dialect of C. Regards, Jörg Höhle

2 1

Re: [cffi-devel] Re: cffi/allegro crash
by Hoehle, Joerg-Cyril 20 Dec '05

20 Dec '05

Hi, Luis Oliveira wrote: >IMHO, one of biggest drawbacks of UFFI (one that I found while testing >uffi-compat on a couple of libs) is the lack of uniformity. For >example, in libs primarily developed on lisps with typed pointers (eg: >SBCL), many (a few?) operations had bogus types and these bugs didn't >show up in these lisps but caused errors on others (and uffi-compat, >with whatever lisp). I don't know whether "uniformity" goes straight to the point. It's clear that UFFI's dual approach is likely to lead to errors. Yet I can understand the design decisions. What I'm missing, is not uniformity but a debugging tool, i.e. one that would match the UFFI given type with the internal SBCL type and throw an error. Then SBCL people would be confident in being able to write portable code. Similarly, I've criticized UFFI for heavy use of EVAL and not clearly specifying what gets evaluated at what time. I think your recent patch to uffi-compat: "Fix parsing of types to match UFFI's behaviour" is not TRT w.r.t. "uniformity". Instead, the type checker I mentioned previously raises errors and allows me to send people bug reports for too many or not enough quotes around UFFI type forms. This becomes especially annoying when writing macros that either analyse or create UFFI forms. Yet, I understand your patch in terms of interoperability, meaning that you want cffi-uffi-compat to accept all the broken SW that the original UFFI accepts as well. That's perfectly fine. Sadly, one HD crash back in July caused me loss of the diffs I had prepared back then to cl-gd, clsql and a few other packages which all contain quoting not in accordance with the UFFI documentation. My checker is in sourceforge's patches section, named "preliminary UFFI code" or so. Regards, Jorg Hohle

1 0

Re: [cffi-devel] Re: cffi/allegro crash
by Hoehle, Joerg-Cyril 20 Dec '05

20 Dec '05

Luis Oliveira writes: >I think the behaviour here >should be uniform since ideally users would test a set of CFFI >bindings on one implementation and be confident that it would work on >the other supported lisps. We all agree on that. However, how do you expect to prevent a long-time Allegro user to inadvertently write 0 instead of (cffi:null-pointer)? This is exactly the same with CLISP+Lispworks with NIL. We don't have static typing in CL, or at least, not without a cost. Strong typing is all we have. I suggest the solution is NOT to add (check-type x ...) but to leave the individual FFI's as is. Allegro must have a reason to stick to integers, even though it's over 10 years that I criticized it the first time :-) BTW, CFFI reinvented the pattern "normal+other-as-different(e.g.NIL)-type" with :string. See foreign-string-to-lisp etc. It's a nice pattern, indeed. Why fight it for :c-pointer, but accept it for :string? But here, I'm not arguing in favour of a NULL->NIL conversion. I'm in favour of leaving the respective implementation's FFI as is, and not generate code for CLISP to turn NIL back into a pointer with a 0 address. I agree that I wouldn't argue with that much heat if this were not related to performance. I believe that if somebody wanted a higher expectation for portability, then s/he might use a "debug converter", which would turn every pointer into a CLOS instance that CFFI would then somehow typecheck at run-time. Obviously, that should be a debug aid only, and not for general code. Similarly, I have a uffi:def-foreign that'll do nothing but type-check at compile (macro expansion) time. I found several bugs in libraries using UFFI this way. Regards, Jörg Höhle.

1 0

Re: [cffi-devel] unnamed callback (closures?)
by Hoehle, Joerg-Cyril 20 Dec '05

20 Dec '05

Martin Simmons wonders: > Jorg> In CLISP, every closure can be turned into a callback, >and this is > Jorg> valuable since it allows to retrieve context >information out of the > Jorg> closure. Is cffi's limitation caused by some implementations? >What are the garbage collection semantics of these closures? >Presumably the >GC can't move them, but when can they be collected? CLISP can not know by itself when callbacks will cease to be invoked by foreign code. Thus they are created with indefinite extent. The programmer can have CLISP deallocate a callback by calling FOREIGN-FREE on the FOREIGN-FUNCTION object. This will cause the little callback struct to be free'd(), CLISP's callback tables to be cleaned up (where it holds the mapping from address to Lisp function object associated with it) and thus the Lisp function becomes eligible for GC as well. In practice this means that the CLISP internal callback table get larger and larger. While CLISP documents the effect of FOREIGN-FREE on FOREIGN-FUNCTION callbacks, I guess nobody but the testsuite uses it. Note that unlike some other implementations, CLISP stores a Lisp function inside the callback, i.e. its (fdefinition 'foo). It does not store the symbol FOO. Therefore, redefinition of FOO will still invoke the old code. BTW, one of my diffs to cffi (yet to be sent) explains why cffi:foreign-free cannot be used to free (cffi:callback 'foo). Regards, Jorg Hohle

1 0

Re: [cffi-devel] Re: cffi/allegro crash
by Hoehle, Joerg-Cyril 20 Dec '05

20 Dec '05

Hi, Luis Oliveira wrote: >Glad you were able to fix this issue! You should pass (cffi:null- >pointer). CLISP does convert NIL to the null pointer which is >unfortunate for this reason (Hi >Joerg!) :-) And it seems Lispworks does too. >The lack of uniformity is very unfortunate here. This needs to be >fixed. We should either a) translate nil->null everywhere, b) don't >translate nil->null anywhere or c) offer the two alternatives through >two different types. Surendra Singhi wrote: >I will vote for clisp's and lispworks approach, it seems least >surprising to me. >Another related issue is when a C function returns a null pointer. The >behavior in that case should be also uniform. My opinion is: a) the pointer type should be opaque to users. b) Users should be pointed at the problematic string <> NIL <> pointer conversion. That's a requirement for documentation of CFFI/UFFI. c) Use of NIL in CFFI or UFFI is declared non-portable in the documentation. d) CFFI (and UFFI) provide means to pass "pointers" that the foreign side will see as a NULL pointer -- they already do so. e) Such a "pointer" may well be represented as NIL on the Lisp side. Nevertheless its opaque. f) CFFI and UFFI must export a type for the pointer types they use that is suitable for use with type declarations. -- a TODO for CFFI Example: (deftype pointer #+allegro number(?) #+clisp(or ffi:foreign-address ffi:foreign-variable ffi:foreign-function null)) g) This type cannot be used for discrimination (by this I mean that the CFFI/UFFI documentation specify this property). That is (typep foo 'pointer) does not mean that foo is nothing but a pointer. It could be an integer in Allegro, etc. h) CFFI does not try to map the implementation's objects into whatever else. Reasons: Meet the user's expectations. Performance. Don't overuse type transforms. i) CFFI exports enough primitives that wrap needed functionality. E.g. POINTER-EQ, IS-NULL-POINTER-P etc. j) Only acceptable exception to h): Provide a new (non-default) pointer type that does null-pointer-p <-> NIL (or whatever else the user chooses) conversion. Such a thing has its uses. Another useful candidate for post-transforms is the "return-type is mostly unsigned int, except for -1" pattern. E.g. read() and the like. -1 is awful as an unsigned int, and using a signed int when its mostly unsigned also feels wrong. You will recognize a bias towards "least common denominator", i.e. UFFI-style. However, I'd really dislike CFFI on CLISP back-check every NIL and create a foreign-pointer with address 0 instead :-( Regards, Jorg Hohle.

3 2

[cffi-devel] [patch] cffi-sys-spec Texinfo idioms
by Stephen Compall 20 Dec '05

20 Dec '05

http://csserver.evansville.edu/~sc87/cffi/cffi-sys-spec.texinfo.1s11.patch is the patch, also attached. http://csserver.evansville.edu/~sc87/cffi/ChangeLog.cffi-sys-spec.texinfo.1… is a changelog, formatted as ChangeLog-luis is in the tarballs directory. Its contents also follow. Please let me know if there is a better/preferred way to do this, particularly as I am unfamiliar with darcs. Mon Dec 19 13:34:00 CST 2005 Stephen Compall <s11(a)member.fsf.org> * CFFI-SYS-SPEC manual updates & Texinfo idioms - Use @var for arguments, rather than @code. - Use @deftpx for a chain of type defns, rather than repeated @deftp blocks. - Use @result{} instead of explicit =>. - Use @r with appropriate formatting for comments in @examples/@lisps. - Introduction: Convert naming convention to a list of general conventions, and define "foreign type". - Rename the pointer functions. - Document new functions. - Use new @impnote macro to add notes, so they can be conditionally removed from the manual. -- Stephen Compall http://scompall.nocandysw.com/blog

2 1

[cffi-devel] Re: CFFI malloc free
by Hoehle, Joerg-Cyril 19 Dec '05

19 Dec '05

Hi, >> If I have a C function which allocates memory for a C string and returns a >> pointer to this memory (return type char *). >> How do I deallocate this memory? Should I use >>`foreign-string-free' or `foreign-free'. >I have been meaning to look into this. Right now, you'd have to use >something like (foreign-funcall "free" :pointer <your-pointer>) I don't understand why there's something unclear here. I thought that's why :string+ptr had been invented. This is exactly the pattern of :string+ptr, and it's embodied in tests/misc-types.lisp:deftest misc-types.string+ptr: a) return string and b) (foreign-free pointer) Is that testcase broken? "Just" non-portable? Well, since foreign-free's specification is to free only things allocated with foreign-alloc, maybe CFFI must export the general C free() as well? The :string+ptr type should then documented this. Note that malloc()/free() is fine for a UNIX shared library. It's somewhat meaningless in the context of a DLL on MS-Windows. But there, one will probably not encounter such a protocol and will have to call the resource free'ing function via the FFI as well. Regards, Jorg Hohle.

2 1

[cffi-devel] what I like about cffi
by Hoehle, Joerg-Cyril 19 Dec '05

19 Dec '05

Hi, Mentions of cffi for CLISP from Yaroslav Kavenchuk and not having heard anything from James Bielman for the last 6 month in the clisp mailing list, I was curious and looked at the current cffi sources. The many short functions make reading the source easy. I also appreciate very much the extensible type system. For example, :string+ptr (returns both foreign pointer and Lisp string) is very useful (esp. with :malloc/free allocation). I wonder whether using that, one can very precisely define the allocation of substructures, which is not possible with the CLISP FFI. What I mean is that :allocation in CLISP guides the whole parameter, and one cannot say: "stack allocate the structure, but malloc the storage for this one slot". It's either take all or nothing. Well, I think I've had 1-2 cases where this would have been useful. BTW, this was also my first use of darcs. I quite enjoy it and appreciate its interactive mode. I like being able to choose among the diff snippets when recording a patch. Is there a means to have darcs show what happened to function foo in file bar (i.e. see the diffs only apropos the current line 100)? Keep on the good work (still some to do :) Regards, Jorg Hohle

2 1

[cffi-devel] CFFI malloc free
by Surendra Singhi 19 Dec '05

19 Dec '05

Hello, If I have a C function which allocates memory for a C string and returns a pointer to this memory (return type char *). Should the return type of this function be :string or :pointer? How do I deallocate this memory? Should I use `foreign-string-free' or `foreign-free'. Is there any way this deallocation can be made automatic? Say, something similar to clisp's `:malloc-free', or should I copy this foreign string as a lisp string and then free the foreign string. Thanks. -- Surendra Singhi http://www.public.asu.edu/~sksinghi/index.html ,---- | WHY SHOULD WE SAVE TIGER? | Ans: Saving the tiger means saving mankind.. | | Help http://pudang.tripod.com/ | or https://secure.worldwildlife.org/forms/tiger_appeal_1.cfm `----

2 4

[cffi-devel] cffi/allegro crash
by Surendra Singhi 19 Dec '05

19 Dec '05

Hello, Any allegro experts around? I wrote cffi bindings (Dec-15) for some functions in wxWidgets, the bindings work smoothly on clisp and lispworks. But they crash on allegro. Then I tried to write the allego bindings directly, but it still crashes. Please help on why this is happening or how to fix it? Thanks. The different bindings and corresponding C code: The dll file which I was using can be found at: http://cvs.sourceforge.net/viewcvs.py/wxcl/wxcl/lib/ P.S.- I am using a Windows XP machine. -- Surendra Singhi http://www.public.asu.edu/~sksinghi/index.html The best-laid plans of mice and men go oft astray.

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

cffi-devel December 2005