[cffi-devel] Re: unnamed callback (closures?)
[resending directly as it seems messages posted through Gmane don't make it to the list :-/ ]
"Luís" == Luís Oliveira <luismbo@gmail.com>: Luís> "Hoehle, Joerg-Cyril" <Joerg-Cyril.Hoehle@t-systems.com> writes: In CLISP, every closure can be turned into a callback, and this is valuable since it allows to retrieve context information out of the closure. Is cffi's limitation caused by some implementations?
Luís> To my knowledge, yes. AFAICT, only SBCL/x86 and CLISP support Luís> that. Luís> Regarding the behaviour of CFFI:DEFCALLBACK as a non-toplevel Luís> form, not only will it set it globally, the callback itself won't Luís> be generated at runtime on most Lisp, IIRC. I've just ran into the same problem. I really really need closures as callbacks. I've tried the naive approach: (defmacro object-event-callback-add (obj type function) `(foreign-funcall "evas_object_event_callback_add" :pointer ,obj callback-type ,(foreign-enum-value 'callback-type type) :pointer (get-callback (defcallback ,(gensym "CB") :void ((data :pointer) (cb-e evas) (cb-obj object) (cb-event :pointer)) (funcall ,function cb-e cb-obj cb-event))) :pointer (null-pointer))) ... but that's a half-baked solution with too many limitations and only works in a simple example that you run once. What are the problems with unnamed callbacks? My guess was that they will never get garbage-collected, but how about storing all of these in a table somewhere and explicitly freeing them after they are no longer needed? In my case I do have a way of finding out when a callback is no longer needed, and I suspect this is the case for many other applications. I mostly develop on SBCL/x86 and ECL, so even if a solution worked on these platforms only, I'd be really happy. --J.
Jan Rychter <jan@rychter.com> writes:
I mostly develop on SBCL/x86 and ECL, so even if a solution worked on these platforms only, I'd be really happy.
If all else failed, you could resort to passing something like: #+sbcl (alien-sap (alien-lambda ...)) #+ecl #| whatever is appropriate for an ecl anon callback |# as a pointer to a CFFI function. While I'd like to see some kind of CALLBACK-LAMBDA as the base primitive in CFFI-SYS for callbacks someday, the implementation support for this is pretty minimal. I think it boils down to: how many Lisps need to support something before we consider adding it an optional feature to CFFI-SYS? If there's enough support, what about exporting some sort of CFFI-FEATURES:NO-ANONYMOUS-CALLBACKS feature and adding CALLBACK-LAMBDA to CFFI-SYS for the Lisps that can do it? James
"James" == James Bielman <jamesjb@jamesjb.com> writes: James> Jan Rychter <jan@rychter.com> writes: I mostly develop on SBCL/x86 and ECL, so even if a solution worked on these platforms only, I'd be really happy.
James> If all else failed, you could resort to passing something like: James> #+sbcl (alien-sap (alien-lambda ...)) +ecl #| whatever is James> #appropriate for an ecl anon callback |# James> as a pointer to a CFFI function. Not sure if I fully understand what you mean here, but I'll try reading some code. Perhaps I can implement something myself. James> While I'd like to see some kind of CALLBACK-LAMBDA as the base James> primitive in CFFI-SYS for callbacks someday, the implementation James> support for this is pretty minimal. James> I think it boils down to: how many Lisps need to support James> something before we consider adding it an optional feature to James> CFFI-SYS? If there's enough support, what about exporting some James> sort of CFFI-FEATURES:NO-ANONYMOUS-CALLBACKS feature and adding James> CALLBACK-LAMBDA to CFFI-SYS for the Lisps that can do it? Well, let me pitch in with a vote: I consider this feature absolutely essential for any serious GUI programming. Generated menus with closures that get called back and already have all the context information are what makes Lisp GUI programming different from C GUI programming. If you don't put that on top of GTK (or EFL, in my case), you end up with a Lisp program that reads like C. So I'd really like to see optional support like what you described above. Besides, I believe that if one or two implementations support a feature like this and it becomes popular, others will follow. Lowest common denominator isn't always the best way to go. --J.
On Wed, 01 Feb 2006 09:46:27 +0100, Jan Rychter <jan@rychter.com> said: "Luís" == Luís Oliveira <luismbo@gmail.com>: Luís> "Hoehle, Joerg-Cyril" <Joerg-Cyril.Hoehle@t-systems.com> writes: In CLISP, every closure can be turned into a callback, and this is valuable since it allows to retrieve context information out of the closure. Is cffi's limitation caused by some implementations?
Luís> To my knowledge, yes. AFAICT, only SBCL/x86 and CLISP support Luís> that.
Luís> Regarding the behaviour of CFFI:DEFCALLBACK as a non-toplevel Luís> form, not only will it set it globally, the callback itself won't Luís> be generated at runtime on most Lisp, IIRC.
I've just ran into the same problem. I really really need closures as callbacks. I've tried the naive approach:
(defmacro object-event-callback-add (obj type function) `(foreign-funcall "evas_object_event_callback_add" :pointer ,obj callback-type ,(foreign-enum-value 'callback-type type) :pointer (get-callback (defcallback ,(gensym "CB") :void ((data :pointer) (cb-e evas) (cb-obj object) (cb-event :pointer)) (funcall ,function cb-e cb-obj cb-event))) :pointer (null-pointer)))
... but that's a half-baked solution with too many limitations and only works in a simple example that you run once.
What are the problems with unnamed callbacks? My guess was that they will never get garbage-collected, but how about storing all of these in a table somewhere and explicitly freeing them after they are no longer needed? In my case I do have a way of finding out when a callback is no longer needed, and I suspect this is the case for many other applications.
The function evas_object_event_callback_add already makes a closure -- on the C side. You can pass arbitrary data as the last argument and it will be passed to your callback, so you can make data an integer index into an array of Lisp closures. Since you know when a callback is no longer needed, you can recycle the ids quite easily. It is easy to wrap this up in a function to avoid the code looking like C every time you add a callback. __Martin
On Wed, 01 Feb 2006 09:46:27 +0100, Jan Rychter <jan@rychter.com> said: "Luís" == Luís Oliveira <luismbo@gmail.com>: Luís> "Hoehle, Joerg-Cyril" <Joerg-Cyril.Hoehle@t-systems.com> writes: In CLISP, every closure can be turned into a callback, and this is valuable since it allows to retrieve context information out of the closure. Is cffi's limitation caused by some implementations?
Luís> To my knowledge, yes. AFAICT, only SBCL/x86 and CLISP support Luís> that.
Luís> Regarding the behaviour of CFFI:DEFCALLBACK as a non-toplevel Luís> form, not only will it set it globally, the callback itself won't Luís> be generated at runtime on most Lisp, IIRC.
I've just ran into the same problem. I really really need closures as callbacks. I've tried the naive approach:
(defmacro object-event-callback-add (obj type function) `(foreign-funcall "evas_object_event_callback_add" :pointer ,obj callback-type ,(foreign-enum-value 'callback-type type) :pointer (get-callback (defcallback ,(gensym "CB") :void ((data :pointer) (cb-e evas) (cb-obj object) (cb-event :pointer)) (funcall ,function cb-e cb-obj cb-event))) :pointer (null-pointer)))
... but that's a half-baked solution with too many limitations and only works in a simple example that you run once.
What are the problems with unnamed callbacks? My guess was that they will never get garbage-collected, but how about storing all of these in a table somewhere and explicitly freeing them after they are no longer needed? In my case I do have a way of finding out when a callback is no longer needed, and I suspect this is the case for many other applications.
The function evas_object_event_callback_add already makes a closure -- on the C side. You can pass arbitrary data as the last argument and it will be passed to your callback, so you can make data an integer index into an array of Lisp closures. Since you know when a callback is no longer needed, you can recycle the ids quite easily.
Right. I was actually trying to avoid using that. Evas designers are smart, so they added the "void *data" pointer that gets passed back to callbacks, but this isn't always the case for all libraries. If it turns out that it isn't possible to have closures as callbacks on the Lisp side, I'll probably do just as you described. Thanks for the suggestion. --J.
I wrote:
> On Wed, 01 Feb 2006 09:46:27 +0100, Jan Rychter <jan@rychter.com> said: > "Luís" == Luís Oliveira <luismbo@gmail.com>: Luís> "Hoehle, Joerg-Cyril" <Joerg-Cyril.Hoehle@t-systems.com> writes: In CLISP, every closure can be turned into a callback, and this is valuable since it allows to retrieve context information out of the closure. Is cffi's limitation caused by some implementations?
Luís> To my knowledge, yes. AFAICT, only SBCL/x86 and CLISP support Luís> that.
Luís> Regarding the behaviour of CFFI:DEFCALLBACK as a non-toplevel Luís> form, not only will it set it globally, the callback itself won't Luís> be generated at runtime on most Lisp, IIRC.
I've just ran into the same problem. I really really need closures as callbacks. I've tried the naive approach:
(defmacro object-event-callback-add (obj type function) `(foreign-funcall "evas_object_event_callback_add" :pointer ,obj callback-type ,(foreign-enum-value 'callback-type type) :pointer (get-callback (defcallback ,(gensym "CB") :void ((data :pointer) (cb-e evas) (cb-obj object) (cb-event :pointer)) (funcall ,function cb-e cb-obj cb-event))) :pointer (null-pointer)))
... but that's a half-baked solution with too many limitations and only works in a simple example that you run once.
What are the problems with unnamed callbacks? My guess was that they will never get garbage-collected, but how about storing all of these in a table somewhere and explicitly freeing them after they are no longer needed? In my case I do have a way of finding out when a callback is no longer needed, and I suspect this is the case for many other applications.
The function evas_object_event_callback_add already makes a closure -- on the C side. You can pass arbitrary data as the last argument and it will be passed to your callback, so you can make data an integer index into an array of Lisp closures. Since you know when a callback is no longer needed, you can recycle the ids quite easily.
Right. I was actually trying to avoid using that. Evas designers are smart, so they added the "void *data" pointer that gets passed back to callbacks, but this isn't always the case for all libraries.
If it turns out that it isn't possible to have closures as callbacks on the Lisp side, I'll probably do just as you described. Thanks for the suggestion.
Ok, after actually trying this out, there are two major problems: 1. This approach works just fine for adding callbacks, but fails for deleting them. The problem is that the C side insists on identifying callbacks to delete by a function pointer. In this approach all callbacks would get the same function pointer on the C side -- that of the dispatching callback function on the Lisp side. Which means I won't be able to delete them on the C side afterwards. 2. I was actually wrong when I said I knew when the callback would be no longer needed. Not quite sure what to do about this yet. I know next to nothing about finalizers -- can I expect them to be supported? Apart from a plea for suggestions, this brings me back to my original question -- can we expect any support for anonymous callbacks (closures as callback functions) in CFFI, at least on SBCL to begin with? --J.
Jan Rychter <jan@rychter.com> writes:
Apart from a plea for suggestions, this brings me back to my original question -- can we expect any support for anonymous callbacks (closures as callback functions) in CFFI, at least on SBCL to begin with?
I'm pretty sure we could support anonymous callbacks, on SBCL and CLISP anyway. Do any other Lisps support this? -- Luís Oliveira luismbo (@) gmail (.) com Equipa Portuguesa do Translation Project http://www.iro.umontreal.ca/translation/registry.cgi?team=pt
"Luís" == Luís Oliveira <luismbo@gmail.com> writes: Luís> Jan Rychter <jan@rychter.com> writes: Apart from a plea for suggestions, this brings me back to my original question -- can we expect any support for anonymous callbacks (closures as callback functions) in CFFI, at least on SBCL to begin with?
Luís> I'm pretty sure we could support anonymous callbacks, on SBCL and Luís> CLISP anyway. Do any other Lisps support this? Quick googling of Lispworks and AllegroCL documentation seems to indicate that in both cases one defines externally callable functions with a macro. No mention of closures as callbacks. That would seem to indicate that at least in these cases it isn't supported, but one would have to ask the vendors to be sure. ECL has si::make-dynamic-callback and would be able to support this, in fact it seems that even defcallback expands to [...] `(si::make-dynamic-callback #'(ext::lambda-block ,name ,arg-names ,@body) ',name ',ret-type ',arg-types ,call-type))) [...] So, perhaps it is worth implementing as an optional feature. Hopefully other implementations will follow suit. --J.
Jan Rychter <jan@rychter.com> writes:
"Luís" == Luís Oliveira <luismbo@gmail.com> writes: Luís> I'm pretty sure we could support anonymous callbacks, on SBCL and Luís> CLISP anyway. Do any other Lisps support this?
So, perhaps it is worth implementing as an optional feature. Hopefully other implementations will follow suit.
I took a shot at implementing this on SBCL and CLISP (I don't have a working ECL installation), here's the patch: This still needs unit tests but cursory testing from the REPL looks good---please give it a try. James
"James" == James Bielman <jamesjb@jamesjb.com> writes: James> Jan Rychter <jan@rychter.com> writes:
> "Luís" == Luís Oliveira <luismbo@gmail.com> writes: Luís> I'm pretty sure we could support anonymous callbacks, on SBCL Luís> and CLISP anyway. Do any other Lisps support this?
So, perhaps it is worth implementing as an optional feature. Hopefully other implementations will follow suit.
James> I took a shot at implementing this on SBCL and CLISP (I don't James> have a working ECL installation), here's the patch: James> This still needs unit tests but cursory testing from the REPL James> looks good---please give it a try. It works for me in SBCL. Neat, thanks! However, I'm puzzled as to why you decided not to name the callback and remember it. Is there really no way to release the memory occupied by the trampoline code and let the closure be GCd when it's no longer needed? In my case, this is going to be a problem. If you generate a bunch of callback-lambdas for each context menu appearance, you will soon end up with lots of them in memory. If there really is no way to release them, I'll probably have to think of a different approach (with dispatching functions, I guess, as Martin suggested). --J.
On Feb 7, 2006, at 12:51 AM, Jan Rychter wrote:
However, I'm puzzled as to why you decided not to name the callback and remember it. Is there really no way to release the memory occupied by the trampoline code and let the closure be GCd when it's no longer needed?
In my case, this is going to be a problem. If you generate a bunch of callback-lambdas for each context menu appearance, you will soon end up with lots of them in memory. If there really is no way to release them, I'll probably have to think of a different approach (with dispatching functions, I guess, as Martin suggested).
Well, the idea was to also add an interface for freeing the callbacks (if that is necessary on a particular Lisp), I just haven't gotten there yet. I don't see how you could free them with finalizers since they won't know about references held by foreign code. James
I wrote:
"James" == James Bielman <jamesjb@jamesjb.com> writes: James> Jan Rychter <jan@rychter.com> writes: >> "Luís" == Luís Oliveira <luismbo@gmail.com> writes: Luís> I'm pretty sure we could support anonymous callbacks, on SBCL Luís> and CLISP anyway. Do any other Lisps support this?
So, perhaps it is worth implementing as an optional feature. Hopefully other implementations will follow suit.
James> I took a shot at implementing this on SBCL and CLISP (I don't James> have a working ECL installation), here's the patch:
James> This still needs unit tests but cursory testing from the REPL James> looks good---please give it a try.
It works for me in SBCL. Neat, thanks!
Are there plans for this to be included in CFFI? If not, why not? --J.
On Sun, 05 Feb 2006 14:41:59 +0100, Jan Rychter <jan@rychter.com> said: Delivered-To: cffi-devel@common-lisp.net
I wrote:
>> On Wed, 01 Feb 2006 09:46:27 +0100, Jan Rychter <jan@rychter.com> said: >> "Luís" == Luís Oliveira <luismbo@gmail.com>: Luís> "Hoehle, Joerg-Cyril" <Joerg-Cyril.Hoehle@t-systems.com> writes: In CLISP, every closure can be turned into a callback, and this is valuable since it allows to retrieve context information out of the closure. Is cffi's limitation caused by some implementations?
Luís> To my knowledge, yes. AFAICT, only SBCL/x86 and CLISP support Luís> that.
Luís> Regarding the behaviour of CFFI:DEFCALLBACK as a non-toplevel Luís> form, not only will it set it globally, the callback itself won't Luís> be generated at runtime on most Lisp, IIRC.
I've just ran into the same problem. I really really need closures as callbacks. I've tried the naive approach:
(defmacro object-event-callback-add (obj type function) `(foreign-funcall "evas_object_event_callback_add" :pointer ,obj callback-type ,(foreign-enum-value 'callback-type type) :pointer (get-callback (defcallback ,(gensym "CB") :void ((data :pointer) (cb-e evas) (cb-obj object) (cb-event :pointer)) (funcall ,function cb-e cb-obj cb-event))) :pointer (null-pointer)))
... but that's a half-baked solution with too many limitations and only works in a simple example that you run once.
What are the problems with unnamed callbacks? My guess was that they will never get garbage-collected, but how about storing all of these in a table somewhere and explicitly freeing them after they are no longer needed? In my case I do have a way of finding out when a callback is no longer needed, and I suspect this is the case for many other applications.
The function evas_object_event_callback_add already makes a closure -- on the C side. You can pass arbitrary data as the last argument and it will be passed to your callback, so you can make data an integer index into an array of Lisp closures. Since you know when a callback is no longer needed, you can recycle the ids quite easily.
Right. I was actually trying to avoid using that. Evas designers are smart, so they added the "void *data" pointer that gets passed back to callbacks, but this isn't always the case for all libraries.
If it turns out that it isn't possible to have closures as callbacks on the Lisp side, I'll probably do just as you described. Thanks for the suggestion.
Ok, after actually trying this out, there are two major problems:
1. This approach works just fine for adding callbacks, but fails for deleting them. The problem is that the C side insists on identifying callbacks to delete by a function pointer. In this approach all callbacks would get the same function pointer on the C side -- that of the dispatching callback function on the Lisp side. Which means I won't be able to delete them on the C side afterwards.
Bleuch, other window systems have managed to do this right. You could overcome this by only installing 1 evas callback and keeping the list of Lisp callbacks entirely on the Lisp side. Remove the evas callback when there are no more Lisp callbacks.
2. I was actually wrong when I said I knew when the callback would be no longer needed. Not quite sure what to do about this yet. I know next to nothing about finalizers -- can I expect them to be supported?
Finalizers are not related to the FFI really, so will be implementation-specific. __Martin
participants (5)
-
James Bielman -
James J Bielman
-
Jan Rychter -
Luís Oliveira -
Martin Simmons