Any idea how safe or not this is? I am going to ask support at Franz.
(defun make-shareable-byte-vector (size) "Create a Lisp vector of SIZE bytes can passed to ;WITH-POINTER-TO-VECTOR-DATA." (ff:allocate-fobject :unsigned-char :lisp size)) ; (defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) "Bind PTR-VAR to a foreign pointer to the data in VECTOR." `(let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,vector))) ,@body))
After some internal discussion, we have come up with this patch, which brings back the foreign-address keyword. The keyword is not a foreign-type and is not used as one.
Pointers passed to or returned from foreign functions use :foreign-address, and there is no second Lisp type. (The original unpatched CFFI incorrectly uses nil as the foreign-type and :foreign-address as the Lisp type.)
This means that you can pass either and address or a Lisp array directly as the pointer.
Note that there cannot be a second type in the pair after the foreign-address keyword, if we want to use this functionality,
It should be safer than the ff:allocate-object method and it looks nicer too.
John Fremlin jf@msi.co.jp writes:
Any idea how safe or not this is? I am going to ask support at Franz.
(defun make-shareable-byte-vector (size) "Create a Lisp vector of SIZE bytes can passed to ;WITH-POINTER-TO-VECTOR-DATA." (ff:allocate-fobject :unsigned-char :lisp size)) ; (defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) "Bind PTR-VAR to a foreign pointer to the data in VECTOR." `(let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,vector))) ,@body))
cffi-devel mailing list cffi-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/cffi-devel
On Tue, Jan 13, 2009 at 8:50 AM, John Fremlin jf@msi.co.jp wrote:
After some internal discussion, we have come up with this patch
Pushed it, thanks!
"Luís Oliveira" luismbo@gmail.com writes:
On Tue, Jan 13, 2009 at 8:50 AM, John Fremlin jf@msi.co.jp wrote:
After some internal discussion, we have come up with this patch
Pushed it, thanks!
Duane at Franz has helpfully come up with a decent way to get the real foreign address from a Lisp array.
(let ((x (make-array 10 :element-type '(unsigned-byte 8) :allocation :static-reclaimable))) (ff:fslot-address-typed :unsigned-char :lisp x))
We will test a modification of with-shareable-byte-vector to use this mechanism for ,ptr-var, hopefully it will work well.
There is still a question of the exact foreign-type we should use (:unsigned-char or :int or a dummy).
On Mon, Jan 19, 2009 at 2:35 AM, John Fremlin jf@msi.co.jp wrote:
Duane at Franz has helpfully come up with a decent way to get the real foreign address from a Lisp array.
(let ((x (make-array 10 :element-type '(unsigned-byte 8) :allocation :static-reclaimable))) (ff:fslot-address-typed :unsigned-char :lisp x))
We will test a modification of with-shareable-byte-vector to use this mechanism for ,ptr-var, hopefully it will work well.
What's the advantage, compared to using :FOREIGN-ADDRESS?
"Luís Oliveira" luismbo@gmail.com writes:
On Mon, Jan 19, 2009 at 2:35 AM, John Fremlin jf@msi.co.jp wrote:
Duane at Franz has helpfully come up with a decent way to get the real foreign address from a Lisp array.
(let ((x (make-array 10 :element-type '(unsigned-byte 8) :allocation :static-reclaimable))) (ff:fslot-address-typed :unsigned-char :lisp x))
We will test a modification of with-shareable-byte-vector to use this mechanism for ,ptr-var, hopefully it will work well.
What's the advantage, compared to using :FOREIGN-ADDRESS?
We can continue to use foreign-address, so that people will be able to (unportably) take advantage of Allegro's generous handling of Lisp arrays.
The advantage of the change is that ptr-var is now actually a foreign pointer as it is in all other cffi-ports.
You might not want to pass the ptr-var directly to a function, but actually use it as a foreign pointer.
For a concrete example, using the recvmsg(2) system call, you pass a structure containing an iovec, which is itself a structure containing the base address and length of the buffer into which the message body should be written.
We want this buffer to be a shareable byte-vector array.
Here is part of a real example
(cffi:with-foreign-objects ((sockaddr-buf :unsigned-char +sockaddr-max-len+) (iov-buf 'iovec) (msghdr-buf 'msghdr) (cmsg-buf :unsigned-char +cmsg-buf-len+)) (cffi-sys:with-pointer-to-vector-data (ptr buf) (cffi:with-foreign-slots ((base len) iov-buf iovec) (setf base ptr len (length buf))) ...
(recvmsg (socket-fd socket) msghdr-buf (logior MSG_DONTWAIT MSG_ERRQUEUE))
))
After light testing and comments from Duane at Franz, this is our proposed shareable-byte-vector implementation for Allegro.
(defun make-shareable-byte-vector (size) "Create a Lisp vector of SIZE bytes which be can passed to WITH-POINTER-TO-VECTOR-DATA." (make-array size :element-type '(unsigned-byte 8) :allocation :static-reclaimable))
(defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) "Bind PTR-VAR to a foreign pointer to the data in VECTOR." `(let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,vector))) ,@body))
It seems to work well.
[...]
John Fremlin jf@msi.co.jp writes:
After light testing and comments from Duane at Franz, this is our proposed shareable-byte-vector implementation for Allegro.
[...]
`(let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,vector)))
Sorry, the testing was not sufficient. This does not work properly with :static-reclaimable buffers, giving a wrong address. We are investigating. . .
[...]
On Mon, Jan 19, 2009 at 5:50 AM, John Fremlin jf@msi.co.jp wrote:
The advantage of the change is that ptr-var is now actually a foreign pointer as it is in all other cffi-ports.
You might not want to pass the ptr-var directly to a function, but actually use it as a foreign pointer.
For a concrete example, using the recvmsg(2) system call, you pass a structure containing an iovec, which is itself a structure containing the base address and length of the buffer into which the message body should be written.
That definitely looks like a legitimate use-case. Let us know when you figure out what the first argument to FF:FSLOT-ADDRESS-TYPED should be.
"Luís Oliveira" luismbo@gmail.com writes:
On Mon, Jan 19, 2009 at 5:50 AM, John Fremlin jf@msi.co.jp wrote:
The advantage of the change is that ptr-var is now actually a foreign pointer as it is in all other cffi-ports.
You might not want to pass the ptr-var directly to a function, but actually use it as a foreign pointer.
For a concrete example, using the recvmsg(2) system call, you pass a structure containing an iovec, which is itself a structure containing the base address and length of the buffer into which the message body should be written.
That definitely looks like a legitimate use-case. Let us know when you figure out what the first argument to FF:FSLOT-ADDRESS-TYPED should be.
Here is the new magic based on fresh clues from Duane . . . This one seems to work. A thorough test suite would be great though!
(defun make-shareable-byte-vector (size) "Create a Lisp vector of SIZE bytes which can be passed to ;WITH-POINTER-TO-VECTOR-DATA." (make-array size :element-type '(unsigned-byte 8) :allocation :static-reclaimable))
(defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) "Bind PTR-VAR to a foreign pointer to the data in VECTOR. Not safe except with vectors allocated by MAKE-SHAREABLE-BYTE-VECTOR and possibly arrays of type simple-array (unsigned-byte 8) (*)." ;;; An array allocated in static-reclamable is a non-simple array in ;;; the normal Lisp allocation area, pointing to a simple array in the ;;; static-reclaimable allocation area. Therefore we have to get out ;;; the simple-array to find the pointer to the actual contents. (with-unique-names (simple-vec) `(excl:with-underlying-simple-vector (,vector ,simple-vec) (let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,simple-vec))) ,@body))))
On Tue, Jan 20, 2009 at 1:43 AM, John Fremlin jf@msi.co.jp wrote:
Here is the new magic based on fresh clues from Duane . . . This one seems to work. A thorough test suite would be great though!
Pushed, thanks!
-
John Fremlin -
Luís Oliveira