cffi-devel

cffi-devel@common-lisp.net

1113 discussions

[cffi-devel] user: cffi in saved core?
by Keith Irwin 07 Jan '06

07 Jan '06

Folks-- I'm the guy with the toy "cl-browser" app which embeds gecko in a gtk app. Some context: What I want to do is create a nice standalone version so people don't have to know implementation details and I therefor (if this goes forward) don't have to write documentation. What this means for me is that I'll create a shell script, save a core with the code in it, save a copy of the runtime, build an installable package, etc, etc, and then, in my script, do something like: /usr/lib/cl-browser/my-sbcl --core /usr/lib/cl-browser/my-core --eval '(browser:main)' So far so good. This kind of thing has worked well for me on other projects. The problem: Alas, but when I try this with sbcl 0.9.7 and cffi (12/30/2005), I get a segmentation fault. Is this because of some problems with CFFI, something inherent in SBCL and its memory model? Here's what happens. sbcl * (require 'browser) * (save-lisp-and-die "foo.core") % sbcl --core foo.core --eval '(browser:main)' ... my debug output printf stuff ... Segmentation fault And that's the end of the story. The browser appears momentarily, then goes away. As near as I can tell, this happens about the time I try connecting callbacks to signals defined in the glade file. On one test of tall this, I got: connecting glade signals <== my printf fatal error encountered in SBCL pid 8097(tid 3085362880): fault in heap page not marked as write-protected So, does this mean we can't save a core with CFFI in it? I know there are other ways I can simulate a standalone executable, but I was just wondering if we users should just take for granted we can't build a new core file (with sbcl, anyway), or is this just an ongoing problem yet to be fixed? Thanks, Keith

2 1

[cffi-devel] DEFINE-FOREIGN-LIBRARY Additions
by James Bielman 07 Jan '06

07 Jan '06

After talking with ignas in #lisp about using this interface for GTK, I came up with a patch to add the following features to DEFINE-FOREIGN-LIBRARY: - A CFFI feature expression of T can be used to define a default clause. - If a library name is of the format (:DEFAULT STRING) then LOAD-FOREIGN-LIBRARY will append an OS-specific library suffix (.so, dll, .dylib, etc) to STRING before trying to load it. (We will need to extend this list as we test on more operating systems...) There is no facility for specifying versions with :DEFAULT; if you need to load a specific version, you must give the full library name. If the library you want to link against is "libfoo.so" on Linux, "libfoo.dylib" on Darwin, and "libfoo.dll" on Windows, then the following should do the right thing: (define-foreign-library foo (t (:default "libfoo"))) I was reluctant to add any default behavior that tries to interpret the foreign library name and add suffixes if not present. I think explicitly requesting addition of the suffix keeps the design cleaner. Luís, I put a patch file up if you'd like to look it over: http://slacknet.com/~jamesjb/suffixes.patch James

2 1

[cffi-devel] New patches: 6-Jan-2006
by loliveira＠common-lisp.net 07 Jan '06

07 Jan '06

Fri Jan 6 09:29:57 CET 2006 James Bielman <jamesjb(a)jamesjb.com> * add :size option to defcunion as with defcstruct M ./src/types.lisp -17 +19 Fri Jan 6 08:47:08 CET 2006 James Bielman <jamesjb(a)jamesjb.com> * add todo list entry about setting union size M ./TODO +1 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-060106.tar.gz

1 0

Re: [cffi-devel] vararg (was: CFFI/CLISP - 2 out of 101 total tes ts failed)
by Hoehle, Joerg-Cyril 06 Jan '06

06 Jan '06

Yaroslav Kavenchuk wrote: >> Oops, if use libintl_sprintf instead of sprintf - all works: >> Hmm. I do not know why sprintf function does not work in clisp/mingw. >if define sprintf as function from msvcrt.dll - all works: Oh boy! >Why there is no opportunity to specify from what library link function? You probably mean in CFFI? UFFI has :module and CLISP has :library >And if I shall link some libraries with identical names of functions - >what to do? "Implementation-dependent" and underspecified On many UNIX systems, the more recently opened libraries probably shadow older ones. Now, do you know in which order the libraries where opened? The general answer is "no". E.g. did you read somewhere in clisp impnotes in what order it's reopening libraries when restarting from an image? Do you know what libraries got loaded in what order, when you're just a library programmer and don't know what the user is doing (and what s/he loaded)? No, you don't. There are several reasons why I prefer programmers to state which library or module thex expect a function to come from. It helps do the right thing. (But the most important reason back in 1995 was that the Amiga had no introspection of shared libraries. You *had* to tell you want e.g. Close from *that* specific library, and it was a reasonably way to avoid conflicts). >I do not know what library contains bad sprintf function. Maybe you can figure it out using (foreign-address #<foreign-function>) and see if you can relate that to the addresses of the .dll loaded into the current process? Regards, Jorg Hohle

6 10

Re: [cffi-devel] protect C frames from Lisp non-local exits in ca llbacks
by Hoehle, Joerg-Cyril 06 Jan '06

06 Jan '06

Stephen Compall wrote: >case of undefined behavior: >(block a > (block b > (unwind-protect (return-from a 1) > (return-from b 2)))) >[1] http://www.lisp.org/HyperSpec/Issues/iss152.html Steele had a nice text about breathing, roaming and fishing crabs in CLtL1 and II. >I would like to hear from those [...] running CFFI This has nothing to do with CFFI. Only the underlying implementation knows the answer. And the general answer is: don't exit non-locally across callbacks! It can always lead to unforeseen trouble. Rationale: a) Consider jumping to a shared library written in say C++, which has its own idea of unwind-protect. If you throw out of this via Lisp, the C++ destructor will never get called. Different run-times don't mix. b) Consider code written in C. It expects a functional style, i.e. it was not written with non-local exits in mind. It may set some variables or worse, allocate resources, call some function (your Lisp callback) and reset that variable upon exit. If you throw out of this via Lisp, the variable will not be reset, the resource onot freed. Your application may cease to work as a consequence of this (been there, done that). And it will take you time to debug and understand why. Of course, if you throw out of C:qsort(), I wouldn't expect trouble. That's no counterproof of the above. But consider calling into .NET (I think Edit Weitz has a binding to .NET). Non-local exits get you immediately out of any specified behaviour. >As an aside: would it be better for the defcallback semantics to simply >say "don't exit non-locally through a C frame"? Exactly. It's the only sensible thing to do. Ideally, every callback should contain something similar to ignore-errors (e.g. X event loops). >It would probably make >callbacks (and perhaps callouts, for that matter) cheaper to call, I don't understand that implication. Regards, Jorg Hohle.

2 1

[cffi-devel] vararg (was: CFFI/CLISP - 2 out of 101 total tests f ailed)
by Hoehle, Joerg-Cyril 06 Jan '06

06 Jan '06

Yaroslav Kavenchuk <kavenchuk(a)jenty.by> writes in cffi-devel: >> clisp from CVS head, cffi-051229.tar.gz >> 2 out of 101 total tests failed: CFFI-TESTS::FUNCALL.VARARGS.DOUBLE, >> CFFI-TESTS::STRUCT.NAMES. >About the FUNCALL.VARARGS.DOUBLE test, it passes on Linux and OSX but >fails on Win32. Yaroslav, I'd be grateful if you could express the testcase in terms of CLISP FFI forms (not CFFI) and submit it to the SF CLISP bugtracker, mentioning varargs in the subject line. So the clisp project would have a bugtracker issue for varargs and a place where to post comments. The current bugtracker FFI issues (long long, sparc64 etc.) are distinct from the specific varargs issue. Still, I maintain that attempts to interface to varargs is going to lead to errors here and there. E.g. I found the following comment in ffcall/avcall.h: :#ifdef rs6000 (applies to MacOS AFAIK) :float/double can be passed in float registers, but we also push them to the :corresponding int registers in case of varargs. I.e. ffcall contains special provision to make varargs work with doubles on PowerPC. It's nice to have, but it need not be there on other systems. And I must retract my recommendation to James to attempt o interface to *v() functions (those that take a va_list*). That's a priori not easier than the other functions. I expect the ffcall package may make this job easier in the future, since basically it itself builds up a similar array internally, so it could just pass a pointer to it to the *v() functions, but that's just a supposition of mine. So far my opinion remains that general infrastructure for varargs is not present in ffcall (despite particular work-around for e.g. PowerPC above), and that alone can explain the mentioned test results: + success on Linux and PowerPC/OSX - failure on Win32 (cygwin? mingw32? MS-VC?) IIRC, someody reported Lispworks explicitly mentions varargs not being supported, and Duane Rettig from Franz reported issues in cll etc. Furthermore, I wonder how a varargs call would look from the Lisp side? While the C compiler knows the args to printf() and can push either int, pointer or double on the stack, what should Lisp code do? A) Do a TYPECASE ((integer character) :int-or-long) ; but what about longlong? (string :string-or-pointer) (float :double via (coerce x 'double-float)) on each argument, at run-time? B) Or, for each call, declare a particular usage, e.g. corresponding to sprintf("%d%s%lld%f") and have the programmer write (foreign-funcall :int :string :long-long :[double-]float) ? That's IMHO still no solution with unknwon (at compile-time) format strings or other such arguments. James Bielman particular problem of interfacing to objc_msgSend[v]() might be solved like this: a) it's only relevant to one architecture (PowerPC/OSX) b) it could restrict itself to either :pointer or :long (ignore :double). Then the the stack layout (and the platform ABI) could be so regular that building the array and calling objc_msgSend() might work across all Lisps. Regards, Jörg Höhle.

2 3

[cffi-devel] Callback Fix Branch
by James Bielman 06 Jan '06

06 Jan '06

Hi all, I've made some progress updating the callback interface for each backend to work with callback pointers in a manner that should be safe when saving/reloading images. The branch is up at: http://slacknet.com/~jamesjb/cffi-callback-fix which can be obtained via 'darcs get', or pulled into an existing repository, or browsed over the web. The only significant visible change is the addition of an exported %CALLBACK function to the CFFI-SYS layer, which returns a pointer for a callback defined by %DEFCALLBACK. CFFI:CALLBACK is implemented in terms of CFFI-SYS:%CALLBACK instead of using the symbol's property list. This should also fix some problems on (at least) OpenMCL where the native callback was defined with a gensym'ed name, which means redefinitions of CFFI callbacks would not actually redefine the native callback. In CMU CL, almost every test fails from a saved image, with or without these changes---I'm not sure what's up with that yet. Allegro, CLISP, OpenMCL, and LispWorks now pass all expected tests from a saved image. (In CLISP, you need to reload foreign libraries by hand after loading the saved image. We might want to add another initialization hook to reload libraries loaded by CFFI...) SBCL passes all tests from a saved image with the PURIFY workaround, and should work out of the box when the underlying callback bug is fixed. I can't get the ECL backend to build, and I don't have access to Corman, so they are untested. Jörg, in the CLISP implementation, I'm a little concerned about using FFI::EXEC-ON-STACK to convert the functions to callbacks, but I couldn't find another way to get the effect of FFI:WITH-C-VAR at runtime, given a parsed type. Comments? James

1 0

[cffi-devel] New patches: 5-Jan-2006
by loliveira＠common-lisp.net 06 Jan '06

06 Jan '06

Thu Jan 5 10:14:44 CET 2006 James Bielman <jamesjb(a)jamesjb.com> * add todo list entry about implementing :uint8 and friends M ./TODO -3 +5 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-060105.tar.gz

1 0

[cffi-devel] Re: cffi: Callbacks and threading issues on LispWorks
by Luís Oliveira 05 Jan '06

05 Jan '06

On 2006-jan-02, at 12:15, Eivind Midtgård wrote: > Dear Mr. Oliveira, > > In November 2004 I inquired to Xanalys about callback support in > LispWorks on Windows and got the following reply: > > - When running a lisp .exe, you can only call back into Lisp in a > thread that was created by Lisp. > - When running a lisp .dll, you can call back into Lisp in any > thread. > > > The first item is an issue not mentioned anywhere in the cffi > documentation. Perhaps it is not important for the most common > usage scenarios. > > > > With best regards, > Eivind Midtgård Eivind, Thanks for pointing this out. We do need to investigate details like this and point them out in the documentation. Happy new year, -- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ Equipa Portuguesa do Translation Project http://www.iro.umontreal.ca/translation/registry.cgi?team=pt

2 1

[cffi-devel] protect C frames from Lisp non-local exits in callbacks
by Stephen Compall 05 Jan '06

05 Jan '06

A brief discussion in #lisp pointed me to the EXIT-EXTENT:MINIMAL ANSI issue[1]. The salient point is that there is no way to portably stop all unwinding for a non-local exit at a particular point, due to this case of undefined behavior: ;;returns 2 under MEDIUM, is error under MINIMAL (block a (block b (unwind-protect (return-from a 1) (return-from b 2)))) A CL implementation that implements MEDIUM should still be fully compliant with the MINIMAL decision. Therefore, I would like to hear from those with particular interest in any particular implementation running CFFI: does implementation X implement EXIT-EXTENT:MEDIUM, and/or does its callback definition form as exploited by cffi-sys:%defcallback have some "magic" unwind stopping? As an aside: would it be better for the defcallback semantics to simply say "don't exit non-locally through a C frame"? It would probably make callbacks (and perhaps callouts, for that matter) cheaper to call, on those without "magic" unwind stopping anyway. [1] http://www.lisp.org/HyperSpec/Issues/iss152.html -- Stephen Compall http://scompall.nocandysw.com/blog

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

cffi-devel