Hello,
I'm kinda new to CFFI.
Is there a reliable/portable way to have lambdas called as a callback from a foreign C function? I assume there's not though.
I could imagine it could be simulated (by a dispatcher callback or something) so I'm wondering if somebody has done something like that before and may share insight into her/his design.
Thanks, Frank
On Wed, Nov 5, 2014 at 7:53 AM, FAU fau@riseup.net wrote:
Is there a reliable/portable way to have lambdas called as a callback from a foreign C function? I assume there's not though.
SBCL has an ALIEN-LAMBDA but, last time I checked, most Lisps don't have a similar feature.
I could imagine it could be simulated (by a dispatcher callback or something) so I'm wondering if somebody has done something like that before and may share insight into her/his design.
I know one of the GTK bindings did something like this, but it used some GTK-specific tricks. IIRC, callbacks were somehow associated with specific GTK objects, and so you could dispatch on those.
If you know the callback will be invoked within your foreign function call, a special variable might do the trick, but you'd have to define callbacks a priori for each and every combination of argument types.
I think you might be able to do this using libffi's closure API, though. (See cffi-libffi for inspiration on how to use libffi.) Do you feel like giving that a go?
On Wed, 2014-11-05 at 09:16 +0000, Luís Oliveira wrote:
On Wed, Nov 5, 2014 at 7:53 AM, FAU fau@riseup.net wrote:
Is there a reliable/portable way to have lambdas called as a callback from a foreign C function? I assume there's not though.
SBCL has an ALIEN-LAMBDA but, last time I checked, most Lisps don't have a similar feature.
I could imagine it could be simulated (by a dispatcher callback or something) so I'm wondering if somebody has done something like that before and may share insight into her/his design.
I know one of the GTK bindings did something like this, but it used some GTK-specific tricks. IIRC, callbacks were somehow associated with specific GTK objects, and so you could dispatch on those.
If you know the callback will be invoked within your foreign function call, a special variable might do the trick, but you'd have to define callbacks a priori for each and every combination of argument types.
I think you might be able to do this using libffi's closure API, though. (See cffi-libffi for inspiration on how to use libffi.) Do you feel like giving that a go?
Thanks for all the details. I'll have a look at this.
On Wed, Nov 5, 2014 at 7:53 AM, FAU fau@riseup.net wrote:
Is there a reliable/portable way to have lambdas called as a callback from a foreign C function? I assume there's not though.
The CLISP FFI has had lambda callbacks since FFI day 1, 20 years ago, as well as other implementations.
The issue with callbacks in CFFI is IMHO one of proper resource deallocation. No one AFAIK has devised a KISS-style lambda callback creation *AND DEALLOCATION* protocol. The reason why deallocation is important is that every lambda callback needs dynamic creation of what the CLISP libffi and GCC call a trampoline. The Lisp system cannot itself determine when the callback trampoline is not needed anymore, so the application MUST tell it.
Various Lisp implementation have baroque restrictions on the number of callbacks (I seem to remember numbers as low as 255 for one). Lambda callback makes it trivial to inadvertently create unnumbered callbacks (e.g. reload your Lisp source during development, run some function in a loop...) as opposed to unique named functions, so the risk of leaking memory is high. The lambda callback APIs in existing Lisp implementations differ a lot. These observations combined make an attempt at portable lambda callbacks a hairy job.
I know one of the GTK bindings did something like this, but it used some GTK-specific tricks.
Can't remember about GTK, I may have LTK in memory, and the simple trick was that lambda was not actually needed. There was a single callback function from the foreign POV, so a unique named function could be used. That Lisp function then did some work to identify the actual Lisp callback and supply it with useful arguments.
Regards, Jörg Höhle
On Mon, Nov 10, 2014 at 8:01 PM, Joerg-Cyril.Hoehle@t-systems.com wrote:
The issue with callbacks in CFFI is IMHO one of proper resource deallocation. No one AFAIK has devised a KISS-style lambda callback creation *AND DEALLOCATION* protocol. The reason why deallocation is important is that every lambda callback needs dynamic creation of what the CLISP libffi and GCC call a trampoline. The Lisp system cannot itself determine when the callback trampoline is not needed anymore, so the application MUST tell it.
http://www.clisp.org/impnotes/dffi.html#dffi-callback-mem suggests that the trampolines for anonymous foreign callbacks are simply leaked. Is that an accurate interpretation?
Luís Oliveira wrote:
http://www.clisp.org/impnotes/dffi.html#dffi-callback-mem suggests that the trampolines for anonymous foreign callbacks are simply leaked. Is that an accurate interpretation?
No, reread-the paragraph starting with: "You can use FFI:FOREIGN-FREE to free the trampoline associated with a FFI:FOREIGN-FUNCTION object, ..."
The paragraph explains that it's not trivial to grab the exact object corresponding to the trampoline. If you don't grab it, it will leak.
Regards, Jörg Höhle
On Tue, Nov 11, 2014 at 5:11 PM, Joerg-Cyril.Hoehle@t-systems.com wrote:
No, reread-the paragraph starting with: "You can use FFI:FOREIGN-FREE to free the trampoline associated with a FFI:FOREIGN-FUNCTION object, ..."
The paragraph explains that it's not trivial to grab the exact object corresponding to the trampoline. If you don't grab it, it will leak.
Got it. Well, thanks for your comments! I think we should be able to do a similar interface using libffi's closure API. It will certainly have the same gotcha's as CLISP's API and it'll be important to document them properly.
Cheers,
-
FAU -
Joerg-Cyril.Hoehle＠t-systems.com -
Luís Oliveira