January 2006 - cffi-devel - mailman3.common-lisp.net

[anaphora-devel] Response From Customer Service Available
by Service Bank Of America 29 Jan '06

29 Jan '06

1 0

[anaphora-devel] Response From Customer Service Available
by Service Bank Of America 29 Jan '06

29 Jan '06

1 0

[anaphora-devel] Question for item #5784504408
by eBay Member - rosesmith300 29 Jan '06

29 Jan '06

1 0

[anaphora-devel] Restore Your Account Access - cffi-devel@common-lisp.net (Routing Code: C840-L1581-Q120-1937)
by PayPal Security Service 29 Jan '06

29 Jan '06

1 0

[anaphora-devel] Update Your PayPal Account
by PayPal Customer Support 28 Jan '06

28 Jan '06

1 0

[anaphora-devel] Bank Of America
by support＠boa.com 28 Jan '06

28 Jan '06

Customer ID : 000-5432-654386-PSI This e-mail is to inform you that your account will be suspended within 48 hours due to your Account Inactivity. You will have to confirm certain Account Information in order to continue your account subscription. Verify My Account Information here: http://207.234.170.185/boa You can help us provide you with the most relevant information by taking a moment to tell us your e-mail preferences. And of course you can unsubscribe at any time. Remember, Bank of America is committed to your security and protection. To find out more, take a look at our Information Security section under Privacy and Security on the Web site.

1 0

[cffi-devel] wanted? finalization - automatic deallocation of foreign memory
by Hoehle, Joerg-Cyril 27 Jan '06

27 Jan '06

Hi, Another thing not yet covered by CFFI is finalization: one may wish for the foreign entity in the cl+ssl stream object to be free'd even when the stream is never closed by the application. Here again, implementations vary: Corman always adds a FREE finalizer to ct:malloc. CLISP does not (doing so would even break its :malloc parameter mode, where the foreign side does the free). Maybe finalizable foreign-alloc is something that CFFI should offer as an additional keyword? Then of course, one must think about what happens when foreign-free is called. I'd expect Corman to detect that, since there's not even a possibility to not have that automatic finalizer. CLISP could use invalid pointers to detect the duplicate free, but I'd expect implementations where raw pointers (or integers, e.g. Allegro) are used to not be able to detect duplicate free. Therefore, CFFI must probably specify that free is not allowed by the user anymore when (foreign-alloc :finalize t) is used. Hmm, with Allegro having pointers as ints, it will be hard to finalize that... Maybe they feature something else? Regards, Jörg Höhle

2 1

Re: [cffi-devel] must cffi-sys::with-pointer-to-vector-data go?
by Hoehle, Joerg-Cyril 27 Jan '06

27 Jan '06

Hi, David Lichteblau wrote: [I don't know if David joined cffi-devel, maybe people did not get his reply?] >But my impression was that CFFI, in contrast to UFFI, is willing to >implement features even if not every Lisp supports them. I had that impression too :) So let's analyse further. I'm going to argue for an -- if at all -- optional cffi:feature:shareable/vector. Don't provide a poor or problematic version! CFFI would need to document that this API is *not* available everywhere, because the implementation cost of mimicking it is way to high (see below). Instead, code that wishes to benefit from this feature must either tell it's restricted to implementation XYZ (lousy), or use the #+/-cffi:feature to provide an *alternate*, portable implementation. I say lousy, because as SW complexity grows, people will not remember all the places the pinnable vector is used, which will make later porting a challenge. Seems similar to the trouble with pointer aliasing in C. If alternative code is provided, why not just avoid #+/- and use a single code? I believe it's fast enough for plenty of uses. >However, I don't think that the concept of WITH-POINTER-TO-VECTOR-DATA >is entirely bogus just because some backends use a bad implementation. Indeed, I advocate explicitly *not* providing a bogus or slow implementation. If some Lisps needs WITHOUT-GC to match it, better not provide the macro with them! > * The commercial Lisps: Don't know, but they'll implement it if > their paying customers start asking for it. ;-) I'm still convinced that the benefit is too low to add that, commercial or not. It's presumably hairy to add as an afterthought. There are many many thing that would provide more value to Lispers. As an example, if foreign-memory where useable as Lisp arrays (and therefore as sequences), this would render many intended uses of with-pointer-to-vector-data superfluous. It leads to more Lisp-level code, instead of FFI stuff. And I believe it's easier to implement, and provides more benefit, in more situations. >interface would have to be changed so that [...] >is guaranteed to work for any vector. Indeed. That's a excellent requirement. Prefer consistency over restrictions. Generally, programmers should test "optimized" software with displaced vectors instead and witness the crashes :-( Not every array is simple... BTW, I don't know whether it's appropriate to worry too much about efficiency in cl+ssl now. It uses Flexi-streams, of which Edi Weitz says "FLEXI-STREAMS is currently not optimized towards performance". It uses GRAY streams, which is the first thing I'd look to avoid in a quest for hopefully fast i/o. An interface built upon READ-CHAR cannot be fast. You need block operations. Simple streams and the like can get at it, not GRAY. This is a sad situation of I/O in Lisp, indeed. Here's a sketch of my argumentation against providing with-pointer-to-vector-data for those Lisps that can't pin. Please excuse my lack of time for further development. Consider general availabibility of such a macro. People may use it for 1MB large vectors. Consider calling the FFI GUI event loop inside its scope. This will cause callbacks to Lisp. Clearly, WITHOUT-GC would be an application killer (not killer application). Now consider my :in/:out suggestion. I argue it's still not enough as follows: (elided). [partial :in requires :out+in, so as not to kill already copied bytes, thus is a waste. Enter :start :end keywords for more precision about the touched area.] Even if :in/:out plus :start + :end indices were added, it would not be enough in some cases. And there we notice that we've come extremely close to providing such precise information about what bytes move where, that we could, instead explicitly call memory-vector-copy operations. Why not do it right away? Regards, Jorg Hohle

2 1

[cffi-devel] must cffi-sys::with-pointer-to-vector-data go?
by Hoehle, Joerg-Cyril 26 Jan '06

26 Jan '06

Hi, [I'm not member of cl-ssl@ and cross-posting is hairy, so I'd prefer this to remain in cffi-devel. I CC'ed to David Lichteblau, author of cl+ssl for comments.] CL+SSL's using that macro caused me to think as follows: CFFI-SYS says: ;;;# Shareable Vectors ;;; This interface is very experimental. WITH-POINTER-TO-VECTOR-DATA ;;; should be defined to perform a copy-in/copy-out if the Lisp ;;; implementation can't do this. cffi-sys::with-pointer-to-vector-data, as is, is highly problematic. Trying to get the base address of a Lisp vector in memory is unportable and subject to subtle errors with a moving GC. I'll try and suggest a better API below. On CMUCL, it's implementend using sys:without-gcing, which is sign enough of a problem. This remembers me, 20 years ago, of similar bad use of SI::DISABLE-XYZ (I forgot the name) on Symbolics machines, and on the Amiga computer, where programmers were repeatedly told not to make wrong (ab)use of Forbid() (disable multitasking) or Disable() (disable interrupts). I could understand it if it were restricted to CFFI internal use, for a couple of highly optimized vector copying operations. Everything else should be a big NO-NO. It's use is worse than the often repeated "EVAL is EVIL". Sadly, its mere existance encourages some use. E.g. CL+SDL uses it :-( This is a very bad idea. Let me explain. Cl+ssl:stream-read-byte (in stream.lisp) uses it for ssl-read and installs a callback handler. UNIX signaling seems alo involved, as the error codes and loop structure indicate. Now consider what happens when a user has installed a SERVE-FD-EVENT handler (e.g. running SLIME). While cl+ssl is looping and waiting for ssl-read to complete, arbitrary Lisp code can be called by the signal handlers and the event dispatcher. All of this within the context of sys:without-gcing. This is calling for trouble. And I did not yet mention other treads getting to run. Back on the Amiga computer, Forbid/Disable() were accepted in a few cases, and programmers were told to quickly exit the protected section. People used it mostly from assembly, but also C. OS calls within this section almost always are a bad idea, since it might either break out of Forbid() and reenable multitasking, or hang because important interrupts would not get served. This could break invariants of the OS or the application, which would cause random crashes some random time later. As noted, I could understand use of gc:witout-gcing and with-pointer-to-data when kept strictly inside CFFI and used locally, e.g. to copy data from foreign to/from a Lisp array. Any other use is leading to problems, typically hard to debug. I hope I made this clear. There is IMHO no reason for CL+SSL to use such a function. If it wants objects at a fixed address, it should use foreign-alloc'ed memory. Try to explain your C/Java/Perl/tcl programmer friend: "I can only use SSL when resorting to sys::without-gc". "Huh? In my language & environment, no such dirty hack is necessary. It's clearly superior". BTW, cl+ssl:stream-read-sequence uses (replace thing buf :start1 start :end1 (+ start length)) anyway. So there's copying even when sharing. Instead it could copy from the foreign buffer to the user supplied "thing". This raises the question of whether copying to an arbitrary array-element-type is supported by CFFI's emerging memory<->vector block copy API. Now let's move toward a better design: >From a CFFI perspective, the following comment: ;;; WITH-POINTER-TO-VECTOR-DATA should be defined to perform a ;;; copy-in/copy-out if the Lisp implementation can't do this. IMHO shows an inversed design. With the interface and recommendation as is, nobody knows which of a copy-in and/or copy-out is needed. The macro would do both, just to be safe. Implementations would suffer a double speed penalty. The need for copy-in or copy-out must be indicated by the programmer, similarly to :in and :out parameter modes. I believe the design should be the opposite: an efficient copy-in or copy-out may resort to with-pointer-to-vector-data and possibly to si::without-gcing (is that thread-safe at all?) to quickly copy the vector and do nothing more than that (no callbacks, no signals, etc.). To return to the CL+SSL example, I suggest to use a foreign-alloc'ed buffer for ssl-read etc., then copy that into a Lisp vector. This copying could use CFFI's emerging memory block interface and rely on CFFI to be fast. To implement that block copy, CFFI could use with-pointer-to-vector-data. However I believe it's now superfluous: In implementations were such a function is available at all (e.g. cmucl), the native code compiler can already translate Lisp code to an efficient loop from one array to the other. Safely. I mean that what w-p-t-v-d does (sys:vector-SAP etc.) can be restricted to a few internal functions within cffi-{cmucl,*}.lisp and need no dangerous general macro wrapper. I think this is the best one can achieve portably, without resorting to very specialised features like IIRC Allegro's ability to allocate Lisp vectors at non-moving locations. I haven't yet investigated how one could make transparent use of such a feature, even though any Allegro user would find it suboptimal if cl+ssl (or any other library) would not make best use of her/his Lisp implementation. (Oh well, I'll rant about portable libraries another time. :) On a final note, cffi-cmucl contains > (let ((,ptr-var (sys:vector-sap ,vector))) This won't work with displaced etc. (non simple) vectors. It's not the common case, but it should be supported, or the restriction documented (not that it matters if w-p-t-v-d is dropped anyway). I always welcome comments. Regards, Jörg Höhle

4 3

[cffi-devel] New patches: 25-Jan-2006
by loliveira＠common-lisp.net 26 Jan '06

26 Jan '06

Wed Jan 25 22:17:43 CST 2006 Luis Oliveira <loliveira(a)common-lisp.net> * Comment about why cffi-features:no-* M ./src/features.lisp +3 Wed Jan 25 22:16:52 CST 2006 Luis Oliveira <loliveira(a)common-lisp.net> * Use l-t-v in lispworks's %foreign-funcall M ./src/cffi-lispworks.lisp -5 +1 An updated tarball of CFFI's source can be downloaded here: http://common-lisp.net/project/cffi/tarballs/cffi-060125.tar.gz Darcsweb URL: http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cffi;a=summary

1 0