[cffi-devel] with-pointer-to-vector-data
I was trying to use Edi Weitz' drakma http client the other day, and found that it depends on cl+ssl which, in turn, uses CFFI's with-pointer-to-vector-data. I run ACL, and with-pointer-to-vector-data is commented out in cffi-allegro. I *believe* that the following definition is what's needed, but have not been able to test it well enough to be sure (I don't really know what might kill it); it was mostly assembled by stumbling through the Franz FF interface documentation. IIUC, we needn't do anything to get a pointer for the C code to use, the important thing is to avoid having the lisp gc move the byte vector around, which is accomplished by the use of register-lisp-value. HTH, R for cffi-allegro.lisp: (defun make-shareable-byte-vector (size) "Create a Lisp vector of SIZE bytes that can be passed to WITH-POINTER-TO-VECTOR-DATA." (make-array size :element-type '(unsigned-byte 8))) ;;; this is a crazy whack at trying to make the shared vectors work. ;;; The support functions just make sure that the vector doesn't get ;;; moved around, so we have a stable pointer, and then I think ;;; passing the vector will Just Work\tm. [2006/09/11:rpg] (defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) "Bind PTR-VAR to a foreign pointer to the data in VECTOR." `(unwind-protect (progn (foreign-functions:register-lisp-value ,vector) (let ((,ptr-var ,vector)) ,@body)) (foreign-functions:unregister-lisp-value ,vector)))
Quoting rpgoldman@sift.info (rpgoldman@sift.info):
a pointer for the C code to use, the important thing is to avoid having the lisp gc move the byte vector around, which is accomplished by the use of register-lisp-value.
Are you sure? I believe r-l-v just returns an integer index into a global table that can later be used to retrieve the object again. That keeps it from being garbage collected, but not from movement in memory. What I am "using" (not well-tested) is this: (defun make-shareable-byte-vector (size) (make-array size :element-type '(unsigned-byte 8) :allocation :static-reclaimable)) (defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) `(let ((,ptr-var ,vector)) ,@body)) (The old discussion of whether this kind of thing is worth supporting at all notwithstanding.) d.
"David" == David Lichteblau <david@lichteblau.com> writes:
David> Quoting rpgoldman@sift.info (rpgoldman@sift.info): >> a pointer for the C code to use, the important thing is to >> avoid having the lisp gc move the byte vector around, which is >> accomplished by the use of register-lisp-value. David> Are you sure? I believe r-l-v just returns an integer David> index into a global table that can later be used to David> retrieve the object again. That keeps it from being David> garbage collected, but not from movement in memory. No, I'm not sure! The somewhat confusing manual materials are (section 8.1 of the ACL manual): "Before accessing a Lisp value from C, it should be registered first. When a Lisp value is registered, an index is returned as a `handle' on the Lisp object. A C function is provided that will return a pointer to the Lisp object given its index. This is preferable to passing addresses of Lisp objects to C functions, since Lisp objects will generally move with each garbage collection. If a garbage collection is triggered from C code - by calling back from C to Lisp - the addresses of Lisp objects originally passed to the C function from Lisp may become invalid. And since one will have lost one's only handle on these Lisp objects, their addresses, there will be no way to find their new addresses. If instead one were to pass the registration indices of these Lisp objects, one could readily find their new addresses using these indices following a call-back to Lisp. .... Note that when one passes Lisp values to foreign functions that have been declared using def-foreign-call, most Lisp data types are converted to corresponding C data types automatically. When one obtains Lisp values by calling lisp_value(), the conversion must be performed explicitly by the foreign code." I read this as saying: 1. I should register the lisp value so that it doesn't get mangled by garbage collection. 2. The use of def-foreign-call means that the indexing stuff will be handled automagically. Either or both of these might be wrong. Best, R
Hi, [Sorry for my coming out of the blue after such a long pause] rpgoldman@sift.info
The somewhat confusing manual materials are (section 8.1 of the ACL manual):
Before accessing a Lisp value from C, it should be registered first. [...] A C function is provided that will return a pointer to the Lisp object given its index. [...] If instead one were to pass the registration indices of these Lisp objects, one could readily find their new addresses using these indices following a call-back to Lisp.
I think this clearly implies that the registered object *can* move if the C code ever uses a callback to Lisp. It's *not* pinned in memory. Scenario Lisp -> C (obtain address) <- Lisp... -> C ;former address can be invalid, ;use index and provided function to obtain new address One can derive from that that if you hold the address for a long time, including a couple of exchanges between Lisp and C, the original address *can* be stale. This gives me just another reason against any sort of "with-pointer-to-vector-data" exposed to the outside world. Use that in tight loops inside lowest-level CFFI code (such low-level that no application programmer ever sees it). Don't use that in application code! As I said long time ago, a good block API is more value to CFFI users than dangerous pinnable vector interfaces. I haven't followed the CFFI evolution this year and don't know if that's been done now. Regards, Jorg Hohle.
participants (3)
-
David Lichteblau -
Hoehle, Joerg-Cyril -
rpgoldman＠sift.info