[cffi-devel] mid-level implementation-specific patches (was: uffi and CUSTOM:* FOREIGN-ENCODING* from clisp)
Hi,
Pascal Bourguignon's proposed patch to src/strings.lisp adds #+clisp, #-clisp to the source. (cf. mailing archives from Mid-November).
I noticed the wide lack of such #+implementation-specific code in the CFFI sources. I like it, but it's limited. I would like to ask what are your thoughts on integration of such code?
I.e., there are a few places (e.g. in strings.lisp) where the CLISP version could clearly benefit from specific code. One of the typical improvements is to get rid of (loop ... do mem-ref #) to copy array contents elementwise and have the FFI built-ins handle them (using native loops). Remember, LOOP compiles to byte code, not tight memcpy()-like code that CMUCL can achieve. BTW, what happened to the proposed block memory interface?
The general idea is that while the low-level API is fine, as it is currently split across each implementation in cffi-xyz.lisp, there's also a need for high-level implementation-specific optimizations. Remember, as Peter Norvig said, a Lisp compiler is not going to help optimizing a Prolog compiler implemented in Lisp. Prolog needs its own optimizations. Same for FFI code.
Regards, Jörg Höhle.
On 2005-dec-09, at 18:22, Hoehle, Joerg-Cyril wrote:
Pascal Bourguignon's proposed patch to src/strings.lisp adds #+clisp, #-clisp to the source. (cf. mailing archives from Mid- November).
I noticed the wide lack of such #+implementation-specific code in the CFFI sources. I like it, but it's limited. I would like to ask what are your thoughts on integration of such code?
James and I have discussed this, the string coded needs a lot of improvement of course. Feature- and optimization-wise.
I.e., there are a few places (e.g. in strings.lisp) where the CLISP version could clearly benefit from specific code. One of the typical improvements is to get rid of (loop ... do mem-ref #) to copy array contents elementwise and have the FFI built-ins handle them (using native loops). Remember, LOOP compiles to byte code, not tight memcpy()-like code that CMUCL can achieve. BTW, what happened to the proposed block memory interface?
Right, once the block memory interface is implemented for all Lisps, we can optimize strings.lisp
Also we need to handle encodings.
On Fri, 2005-12-09 at 19:22 +0100, Hoehle, Joerg-Cyril wrote:
Pascal Bourguignon's proposed patch to src/strings.lisp adds #+clisp, #-clisp to the source. (cf. mailing archives from Mid-November).
I noticed the wide lack of such #+implementation-specific code in the CFFI sources. I like it, but it's limited. I would like to ask what are your thoughts on integration of such code?
I.e., there are a few places (e.g. in strings.lisp) where the CLISP version could clearly benefit from specific code. One of the typical improvements is to get rid of (loop ... do mem-ref #) to copy array contents elementwise and have the FFI built-ins handle them (using native loops). Remember, LOOP compiles to byte code, not tight memcpy()-like code that CMUCL can achieve. BTW, what happened to the proposed block memory interface?
The general idea is that while the low-level API is fine, as it is currently split across each implementation in cffi-xyz.lisp, there's also a need for high-level implementation-specific optimizations. Remember, as Peter Norvig said, a Lisp compiler is not going to help optimizing a Prolog compiler implemented in Lisp. Prolog needs its own optimizations. Same for FFI code.
I think the approach I would like to take to handle this situation is to add a function to the backend for converting foreign strings to/from Lisp that can be aware of encoding, as well as implement the block memory interface.
Another idea I just had---if we run into more cases where optimizing high-level functions for CLISP is a big performance win, but it doesn't make sense to add new backend functions, how about adding "cffi/src/optimizers-clisp.lisp" (and associated files for other implementations) that use compiler macros to perform the optimization?
I know these aren't guaranteed to be expanded but so far in my CFFI experience they seem to be expanded when I want them. I really would like to avoid reader conditionals in the portable source as much as possible.
My time for CFFI hacking is limited these days (alas) but I will try to take a look at implementing this soon.
James
-
Hoehle, Joerg-Cyril -
James Bielman -
Luís Oliveira