"Luís Oliveira" <luismbo@gmail.com> writes:

On Tue, Jan 13, 2009 at 8:50 AM, John Fremlin <jf@msi.co.jp> wrote:

...

After some internal discussion, we have come up with this patch

Pushed it, thanks!

Duane at Franz has helpfully come up with a decent way to get the real foreign address from a Lisp array. (let ((x (make-array 10 :element-type '(unsigned-byte 8) :allocation :static-reclaimable))) (ff:fslot-address-typed :unsigned-char :lisp x)) We will test a modification of with-shareable-byte-vector to use this mechanism for ,ptr-var, hopefully it will work well. There is still a question of the exact foreign-type we should use (:unsigned-char or :int or a dummy).

"Luís Oliveira" <luismbo@gmail.com> writes:

On Mon, Jan 19, 2009 at 2:35 AM, John Fremlin <jf@msi.co.jp> wrote:

...

Duane at Franz has helpfully come up with a decent way to get the real foreign address from a Lisp array.

(let ((x (make-array 10 :element-type '(unsigned-byte 8) :allocation :static-reclaimable))) (ff:fslot-address-typed :unsigned-char :lisp x))

We will test a modification of with-shareable-byte-vector to use this mechanism for ,ptr-var, hopefully it will work well.

What's the advantage, compared to using :FOREIGN-ADDRESS?

We can continue to use foreign-address, so that people will be able to (unportably) take advantage of Allegro's generous handling of Lisp arrays. The advantage of the change is that ptr-var is now actually a foreign pointer as it is in all other cffi-ports. You might not want to pass the ptr-var directly to a function, but actually use it as a foreign pointer. For a concrete example, using the recvmsg(2) system call, you pass a structure containing an iovec, which is itself a structure containing the base address and length of the buffer into which the message body should be written. We want this buffer to be a shareable byte-vector array. Here is part of a real example (cffi:with-foreign-objects ((sockaddr-buf :unsigned-char +sockaddr-max-len+) (iov-buf 'iovec) (msghdr-buf 'msghdr) (cmsg-buf :unsigned-char +cmsg-buf-len+)) (cffi-sys:with-pointer-to-vector-data (ptr buf) (cffi:with-foreign-slots ((base len) iov-buf iovec) (setf base ptr len (length buf))) ... (recvmsg (socket-fd socket) msghdr-buf (logior MSG_DONTWAIT MSG_ERRQUEUE)) ))

John Fremlin <jf@msi.co.jp> writes:

After light testing and comments from Duane at Franz, this is our proposed shareable-byte-vector implementation for Allegro. [...] `(let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,vector)))

Sorry, the testing was not sufficient. This does not work properly with :static-reclaimable buffers, giving a wrong address. We are investigating. . . [...]

"Luís Oliveira" <luismbo@gmail.com> writes:

On Mon, Jan 19, 2009 at 5:50 AM, John Fremlin <jf@msi.co.jp> wrote:

...

The advantage of the change is that ptr-var is now actually a foreign pointer as it is in all other cffi-ports.

You might not want to pass the ptr-var directly to a function, but actually use it as a foreign pointer.

For a concrete example, using the recvmsg(2) system call, you pass a structure containing an iovec, which is itself a structure containing the base address and length of the buffer into which the message body should be written.

That definitely looks like a legitimate use-case. Let us know when you figure out what the first argument to FF:FSLOT-ADDRESS-TYPED should be.

Here is the new magic based on fresh clues from Duane . . . This one seems to work. A thorough test suite would be great though! (defun make-shareable-byte-vector (size) "Create a Lisp vector of SIZE bytes which can be passed to ;WITH-POINTER-TO-VECTOR-DATA." (make-array size :element-type '(unsigned-byte 8) :allocation :static-reclaimable)) (defmacro with-pointer-to-vector-data ((ptr-var vector) &body body) "Bind PTR-VAR to a foreign pointer to the data in VECTOR. Not safe except with vectors allocated by MAKE-SHAREABLE-BYTE-VECTOR and possibly arrays of type simple-array (unsigned-byte 8) (*)." ;;; An array allocated in static-reclamable is a non-simple array in ;;; the normal Lisp allocation area, pointing to a simple array in the ;;; static-reclaimable allocation area. Therefore we have to get out ;;; the simple-array to find the pointer to the actual contents. (with-unique-names (simple-vec) `(excl:with-underlying-simple-vector (,vector ,simple-vec) (let ((,ptr-var (ff:fslot-address-typed :unsigned-char :lisp ,simple-vec))) ,@body))))