January 2012 - cffi-devel - mailman3.common-lisp.net

[cffi-devel] how to treat expected failures in tests
by Anton Vodonosov 14 Mar '12

14 Mar '12

Hello. For my common lisp testing project I aggregate results of a library test suite into single value - ok/fail. I just tested ECL and have the following output from CFFI test suite: 4 out of 228 total tests failed: DEFCFUN.NOOP, CALLBACKS.BFF.1, STRING.ENCODING.UTF-16.BASIC, STRING.ENCODINGS.ALL.BASIC. No unexpected failures. What meaning do you put into the term "expected failure"? Does it mean the library is buggy, but these bugs are known? Or it means that some non-required features are absent, but the library in general OK? I am interested in both short answer - as a library author, how do you think CFFI test suite should be marked if only expected failures present - OK or FAIL? And also I am curious in this concrete example, what these 4 failures mean for CFFI on ECL? Best regards, - Anton

10 25

[cffi-devel] Digitool MCL support for CFFI (was Re: [info-mcl] ? cffi)
by Chun Tian (binghe) 10 Mar '12

10 Mar '12

Hi, CFFI developers Digitool MCL was a old&famous Common Lisp implementation on PPC Mac OS. Recently it was ported into Intel Mac OS X by running in Rosetta (a PPC emulator by Apple). So there're people still using this platform even when Clozure CL is a better choice. Attach is two patches, one is a CFFI MCL support patch made by James Anderson, the other is a trivial patch for MCL support of trivial-features made by me. With these two patch, CFFI can now be loaded into Digitool MCL. This work seems work with James' own project which is based on CFFI. But CFFI-TESTS cannot run at this time, because ASDF:RUN-SHELL-COMMAND doesn't support MCL (I don't know how to fix this right now). IOlib also cannot work in MCL because Gray Streams wasn't supported by MCL yet ... However, I hope CFFI maintainer could merge this patch and let other MCL users help improving exist work. Chun Tian (binghe) 在 2010-9-28，06:41， james anderson 写道： > good morning; > > i have worked out enough of a cffi-mcl to be able to accomplish the immediate goal - to use the cl-zmq zeromq ffi binding.[1] > in case someone wants to carry it further, it's posted along with a mcl-compatible zmq version,[2] which eliminates un-ported dependencies. > > --- > [1] : http://repo.or.cz/w/cl-zmq.git > [2] : http://github.com/lisp/de.setf.zeromq > > On 2010-09-22, at 08:29 , Chun Tian (binghe) wrote: > >> I've read these documents. MCL 4.0's FFI seems based on the function "define-entry-point" which then depends on "ppc-ff-call", both of them are still available in MCL 5.x, and the related MCL source code is "lib/deftrap.lisp", which was last modified in 2005. >> >> However, I think "define-entry-point" is not the preferred way any more: >> >> ; Change log >> ; >> ; put back define-entry-point and things it depends on - though MCL doesn't use it users may >> ; ------ 5.2b2 >> >> I think CFFI people, if want to support MCL, should research "deftrap.lisp" and figure out the preferred way to do the work. But for me, a USOCKET maintainer, I'd like to use this interface directly to implement UDP networking, which was lack from OpenTransport. >> >> --binghe >> >> 在 2010-9-22，13:30， james anderson 写道： >> >>> there are two sources. >>> >>> if you go to ftp.clozure.com, you will find the 5.2 release and a copy of the documentation for 4.0. >>> i know of no revised documentation for 5.2. other than that, there is always the source. >>> i believe 5.2 has a very different calling mechanism then 4.0, as it uses a different os x facility to manage foreign libraries. >>> >>> the biggest '?' is turning off the gc for the dynamic extent of any foreign calls which pass lisp data. i do not know if that is possible in mcl, but it could be faked by moving the data to/from temporary external storage just to get it working. >>> two little question marks are the data type size calculations and the call-back protocol. >>> since the source is there, there's nothing stopping me from adapting the clozure-specific code to work in mcl. >>> i was just hoping to avoid having to wrap my head around those bits. >>> >>> >>> On 2010-09-22, at 04:45 , Chun Tian (binghe) wrote: >>> >>>> Never know how to do FFI in Digitool MCL ... Where can I find some API documents? >>>> >>>> --binghe >>>> >>>> 在 2010-9-22，05:36， james anderson 写道： >>>> >>>>> good morning; >>>>> >>>>> has anyone ported cffi to mcl? >>>>> >>>>> >>>>> _______________________________________________ >>>>> info-mcl mailing list >>>>> info-mcl(a)clozure.com >>>>> http://clozure.com/mailman/listinfo/info-mcl >>>> >>>> _______________________________________________ >>>> info-mcl mailing list >>>> info-mcl(a)clozure.com >>>> http://clozure.com/mailman/listinfo/info-mcl >>> >>> _______________________________________________ >>> info-mcl mailing list >>> info-mcl(a)clozure.com >>> http://clozure.com/mailman/listinfo/info-mcl >> >> _______________________________________________ >> info-mcl mailing list >> info-mcl(a)clozure.com >> http://clozure.com/mailman/listinfo/info-mcl > > _______________________________________________ > info-mcl mailing list > info-mcl(a)clozure.com > http://clozure.com/mailman/listinfo/info-mcl

2 3

Re: [cffi-devel] First stab at reviewing cffi-fsbv
by Liam Healy 11 Feb '12

11 Feb '12

Hi Luis, On Sun, Dec 11, 2011 at 8:36 PM, Luís Oliveira <luismbo(a)gmail.com> wrote: > Hello, > > I went through all the code and here are my review notes. (In org-mode > format.) I suggest we tackle them incrementally, not necessarily in > this order. Meanwhile, I'll be tackling the missing bits in the new > convert-into-foreign-memory. Should we be having this discussion in > cffi-devel btw? > Sure. I've copied the list for anyone who's interested. > * cffi-fsbv.asd > What happens on #-unix? > (:cffi-grovel-file "libffi" :pathname #+unix "libffi-unix") > Absolutely nothing. Someone named "CRLF0710" sent me an email claiming to have a libffi-win32 for the old standalone version of FSBV, and I wrote back saying I was interested but I've heard nothing. > * fsbv/package.lisp > 1. Regarding SIZET, I'd say we can define this type using a > keyword :SIZET. > Yes, if you think that's OK. That was me totally sneaking something in that GSLL needed that was awkward to generate there but that CFFI could generate easily. > 2. cffi-fsbv calls and hooks into so many of CFFI internals. Should > we just use the CFFI package? I mean, the code is full of ::s. > I'm fine with that. I don't think there'll be any symbol conflicts. > > * fsbv/libffi-unix.lisp > 1. OSX ships with libffi. Is (cc-flags "-I/opt/local/include/") > necessary? (I guess it doesn't hurt. Just curious. Perhaps it > didn't ship with libffi in the past?) > Ugh, OSX. It apparently does ship with libffi, except when it doesn't. There are so many ways to install system software, I just gave up. I'm all for your approach because it does simplify the code and make it match the other OSes; when the complaints come in to cffi-devel, you can answer them :-) > > 2. Aren't :uint and :ushort equivalent to: > (ctype ushort "unsigned short") > (ctype unsigned "unsigned")? > Um, I don't know? What are you advocating here? Removing something that I put in? :uint and :ushort don't ring a bell for me. > > 3. Do we really need to grovel this stuff? Grovelling requires a C > compiler which is particularly tricky on Windows. It'd be nice to > avoid that requirement if possible. Didn't see anything in here > that seemed to *require* grovelling. Did I miss something? > "Here" means libffi-unix? The constants at the end aren't necessary, but I think the rest is. In particular, FFI_DEFAULT_ABI (abi) and FFI_OK (status) are pretty critical, as is the struct ffi-type. I'm a little confused; I thought the point of cffi-grovel was to provide access to definitions in .h files, and that's what we need for ffi.h and ffi-target.h. How do you propose defining these without groveling? Just hardwiring into lisp and hoping no one ever changes those values? By the way I have FFI_STDCALL commented out and didn't try to mate it with CFFI's definition on Windows, perhaps CRLF0710 or another Windows user can figure that out. > * fsbv/build-in-type.lisp > 1. Could we use DEFCVAR here to declare the various ffi_type_* > globals? (I think it'd be slightly clearer.) > I kind of like the automatic way of doing it; manually means that every time something changes (new type added or old one removed) you have to remember to do it in two places. That's a recipe for breakage. > 2. Why do we have to cache pointers for FOREIGN-TYPE-ALIAS and > FOREIGN-ENUM? (But see my comments on fsbv/cstruct.lisp) > We probably don't. > 3. Why different LIBFFI-TYPE-POINTER for those two types? > FOREIGN-ENUM inherits from FOREIGN-TYPE-ALIAS. > Is it really different, or is it just getting the pointer from the underlying type? > * fsbv/cstruct.lisp > 1. slots-in-order should be moved near FOREIGN-STRUCT-TYPE. > 2. I think we should move libffi-type-pointer out of FOREIGN-TYPE > and centralize storage in an hash-table. > a) this simplifies implementation a little bit: it's weird that > pretty much all libffi-type-pointer methods have :AROUND > qualifiers. > b) makes it easy to avoid memory leaks when redefining/reloading > structure types. (Although this sort of leak is probably not a > huge issue, it's certainly inelegant.) > 3. The code that creates ffi_types for structs could use some > refactoring. :-) I'll give it a go. > Sure. > > * fsbv/functions.lisp > 1. I think this file can benefit from a little bit of refactoring. > 2. I *think* we can move the indirection logic from the translation > methods onto FFCALL-BODY-LIBFFI, but it's not yet clear how that > can happen. > Interesting thought, I'd like to hear your ideas. > > * fsbv/example.lisp > 1. Need to clear the dead code and move it to examples/. > examples.lisp? It's probably all dead code now, just get rid of the whole thing. > > * src/functions.lisp > 1. We can add a neat restart to *foreign-structures-by-value*. > Yes I think you mentioned this idea on the todo list. I expect the restart is "load cffi-libffi"? > > * src/structures.lisp > 1. As I've mentioned before, we should discuss how we can implement > this sort of functionality in a backwards-compatible way. > I'd prefer not defaulting to previous assumptions (that is, no translation of structures), if that's what backwards-compatible means. But I'm interested in how you'd approach this. * src/early-types.lisp > 1. missing expand-into-foreign-memory, and all the hooking up that > implies. Started working on that. > 2. I think the :indirect keyword is delegating the responsibility to > the wrong place. All the type translations (in CFFI or in other > libraries) would have to be updated accordingly! As mentioned in > the fsbv/functions.lisp notes, I think we can handle the > indirection there. (It might require a slightly different way of > hooking up ffcall-body-libffi with foreign-funcall.) > No doubt. I would like to avoid special-casing though, e.g. "here's an enum, got to indirect it". I think that kind of thing belongs in a method associated with the type. > * Nomenclature > I can forsee using libffi to deal with stuff like callbacks, long > long, varargs, etc. Should we rename cffi-fsbv to cffi-libffi? > Sure. Also, you mentioned that some implementations have their own call-by-value mechanism, so it makes more sense to rename it, since they would have "fsbv" but no libffi. > > * Wishlist > *** foreign-funcall-varargs and friends could use libffi > ... to do proper varargs. > *** defcallbacks with structures... > Yup. Liam > > -- > Luís Oliveira > http://r42.eu/~luis/ >

1 1

[cffi-devel] Error on ABCL: Unrecognized keyword argument :CONVENTION
by Anton Vodonosov 26 Jan '12

26 Jan '12

Hello. When I run the following command: java -cp jna.jar;C:\Users\anton\unpacked\abcl\abcl-bin-1.0.0\abcl.jar org.armedbear.lisp.Main --batch --eval "(ql:quickload :drakma)" an error occurs: Armed Bear Common Lisp 1.0.0-svn-13663 Java 1.6.0_26 Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.967 seconds. Startup completed in 3.725 seconds. Loading C:\Users\anton\.abclrc completed in 17.21 seconds. To load "drakma": Load 1 ASDF system: drakma ; Loading "drakma" .............. Caught PROGRAM-ERROR while processing --eval option "(ql:quickload :drakma)": Unrecognized keyword argument :CONVENTION I think it's cffi error (cffi is the only library having the :convention in source code). As you see in my command line, i load drakma. Drakma depends on cffi via cl+ssl. If I do (ql:quickload :cffi) it works, but I assume the error happens during macro-expansion of some CFFI macro, that's why loading is not enough, some code using cffi should be loaded too. Best regards, - Anton

2 1

[cffi-devel] cffi:load-foreign-library error on Clozure Cl
by bob 24 Jan '12

24 Jan '12

Hi, I am not able to load any dll library using the cffi:load-foreign-library function. I have tried using working dlls libcairo.dll, zlib1.dll, .. all of which have been working perfectly in other softwares. I have tried using win32 as well as win64 versions. This is the trace: Unable to load foreign library (LIBRARY-205). Error opening shared library libpng14-14.dll : %1 is not a valid Win32 application. . [Condition of type CFFI:LOAD-FOREIGN-LIBRARY-ERROR] Restarts: 0: [RETRY] Try loading the foreign library again. 1: [USE-VALUE] Use another library instead. 2: [RETRY] Retry SLIME REPL evaluation request. 3: [*ABORT] Return to SLIME's top level. 4: [ABORT-BREAK] Reset this thread 5: [ABORT] Kill this thread Backtrace: 0: (CFFI::FL-ERROR "Unable to load foreign library (~A).~% ~A" #:LIBRARY-205 "Error opening shared library libpng14-14.dll : %1 is not a valid Win32 application. .") 1: ((:INTERNAL CFFI::%DO-LOAD CFFI::%DO-LOAD-FOREIGN-LIBRARY) #<CFFI:FOREIGN-LIBRARY #x210144BFED> #:LIBRARY-205 "libpng14-14.dll") 2: (CFFI:LOAD-FOREIGN-LIBRARY "libpng14-14.dll" :SEARCH-PATH NIL) 3: (CCL::CALL-CHECK-REGS CFFI:LOAD-FOREIGN-LIBRARY "libpng14-14.dll") 4: (CCL::CHEAP-EVAL (CFFI:LOAD-FOREIGN-LIBRARY "libpng14-14.dll")) 5: (SWANK::EVAL-REGION "(cffi:load-foreign-library \"libpng14-14.dll\")\n") 6: ((:INTERNAL SWANK::REPL-EVAL)) 7: (SWANK::TRACK-PACKAGE #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::REPL-EVAL) #x21012B1FEF>) 8: (SWANK::CALL-WITH-RETRY-RESTART "Retry SLIME REPL evaluation request." #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::REPL-EVAL) #x21012B206F>) 9: (SWANK::CALL-WITH-BUFFER-SYNTAX NIL #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::REPL-EVAL) #x21012B20AF>) 10: (SWANK::REPL-EVAL "(cffi:load-foreign-library \"libpng14-14.dll\")\n") 11: (CCL::CALL-CHECK-REGS SWANK:LISTENER-EVAL "(cffi:load-foreign-library \"libpng14-14.dll\")\n") 12: (CCL::CHEAP-EVAL (SWANK:LISTENER-EVAL "(cffi:load-foreign-library \"libpng14-14.dll\")\n")) 13: (SWANK:EVAL-FOR-EMACS (SWANK:LISTENER-EVAL "(cffi:load-foreign-library \"libpng14-14.dll\")\n") "COMMON-LISP-USER" 38) 14: (SWANK::PROCESS-REQUESTS NIL) 15: ((:INTERNAL SWANK::HANDLE-REQUESTS)) 16: ((:INTERNAL SWANK::HANDLE-REQUESTS)) 17: (SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK #<Compiled-function SWANK:SWANK-DEBUGGER-HOOK #x21007251EF> #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::HANDLE-REQUESTS) #x2100AB3BEF>) 18: (SWANK::CALL-WITH-BINDINGS ((*STANDARD-OUTPUT* . #<SWANK-BACKEND::SLIME-OUTPUT-STREAM #x2100AB2A7D>) (*STANDARD-INPUT* . #<SWANK-BACKEND::SLIME-INPUT-STREAM #x2100AB2E3D>) ..))) #<CCL:COMPILED-LEXICAL.. 19: (SWANK::HANDLE-REQUESTS #<CONNECTION #x210099BC7D> NIL) 20: (CCL::RUN-PROCESS-INITIAL-FORM #<PROCESS repl-thread(10) [Active] #x2100AA1CCD> (#<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x2100AA1A6F>)) 21: ((:INTERNAL (CCL::%PROCESS-PRESET-INTERNAL (CCL:PROCESS))) #<PROCESS repl-thread(10) [Active] #x2100AA1CCD> (#<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x2100AA1A6F>)) 22: ((:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION)) Should it be 'cause I am using x64 version of Clozure Cl ?

3 2

[cffi-devel] ELS2012 Zadar, Croatia, Call for Papers
by Marco Antoniotti 23 Jan '12

23 Jan '12

Apologies for the multiple postings... =========================================================================== European Lisp Symposium 2012, Zadar, Croatia, April 30th - May 1st, 2012 http://european-lisp-symposium.org The purpose of the European Lisp Symposium is to provide a forum for the discussion and dissemination of all aspects of design, implementation and application of any of the Lisp and Lisp-inspired dialects, including Common Lisp, Scheme, Emacs Lisp, AutoLisp, ISLISP, Dylan, Clojure, ACL2, ECMAScript, Racket, SKILL, and so on. We encourage everyone interested in Lisp to participate. The main theme of the 2012 European Lisp Conference is "Interoperabilty: Systems, Libraries, Workflows". Lisp based and functional-languages based systems have grown a variety of solutions to become more and more integrated with the wider world of Information and Communication Technologies in current use. There are several dimensions to the scope of the solutions proposed, ranging from "embedding" of interpreters in C-based systems, to the development of abstractions levels that facilitate the expression of complex context dependent tasks, to the construction of exchange formats handling libraries, to the construction of theorem-provers for the "Semantic Web". The European Lisp Symposium 2012 solicits the submission of papers with this specific theme in mind, alongside the more traditional tracks which have appeared in the past editions. We invite submissions in the following forms: Papers: Technical papers of up to 15 pages that describe original results or explain known ideas in new and elegant ways. Demonstrations: Abstracts of up to 4 pages for demonstrations of tools, libraries, and applications. Tutorials: Abstracts of up to 4 pages for in-depth presentations about topics of special interest for at least 90 minutes and up to 180 minutes. Lightning talks: Abstracts of up to one page for talks to last for no more than 5 minutes. All submissions should be formatted following the ACM SIGS guidelines and include ACM classification categories and terms. For more information on the submission guidelines and the ACM keywords, see: http://www.acm.org/sigs/publications/proceedings-templates and http://www.acm.org/about/class/1998. Important dates: Jan 31st 2012: submission deadline Feb 21st 2012: acceptance results April 30th, 2012 Conference opens Program Commitee. Chair: Marco Antoniotti, Università degli Studi di Milano Bicocca, Milan, ITALY Local organizers: Damir Ćavar, Eastern Michigan University Franjo Pehar, University of Zadar Damir Kero, University of Zadar Members: Giuseppe Attardi, Università degli Studi di Pisa, Pisa, ITALY Pascal Costanza, Intel, Bruxelles, BELGIUM Marc Feeley, Université de Montreal, Montreal, CANADA Scott McKay, Google, U.S.A. Kent Pitman, U.S.A. Christophe Rhodes, Department of Computing, Goldsmiths, University of London, London, UNITED KINGDOM Robert Strandh, LABRI, Université de Bordeaux, Bordaux, FRANCE Didier Verna, EPITA / LRDE, FRANCE Taiichi Yuasa, Kyoto University, JAPAN -- Marco Antoniotti

1 0

[cffi-devel] Add complex aggregate objects support
by Клочков Роман 22 Jan '12

22 Jan '12

I try to make smart struct support in my project. It works. Now i can have foreign struct in lisp as lisp structures (defclass/defstruct). The only problem for me is mem-ref. When there is an array of structs, I need to use something like cffi::aggregatep to set fact, that struct are positioned in row, not pointers to them. But in that case I cannot set translator, because in cffi::mem-ref 110 (if (aggregatep ptype) 111 (inc-pointer ptr offset) 112 (let ((ctype (canonicalize ptype))) 113 #+cffi-sys::no-long-long 114 (when (or (eq ctype :long-long) (eq ctype :unsigned-long-long)) 115 (return-from mem-ref 116 (translate-from-foreign (%emulated-mem-ref-64 ptr ctype offset) 117 ptype))) 118 ;; normal branch 119 (translate-from-foreign (%mem-ref ptr ctype offset) ptype))))) Can we change (inc-pointer ptr offset) to (translate-from-foreign (inc-pointer ptr offset) ptype) ? If it is really pointer it remains unchanged, but if translate-from is overrided for particular type one can get any information out, not just pointers. -- Клочков Роман

2 1

[cffi-devel] Unit tests failures on different lisps
by Anton Vodonosov 06 Jan '12

06 Jan '12

Hello. I am running tests of some most often downloaded CL libraries, including CFFI. So far CFFI was only tested on Linux. It has Linux, has different number (of expected and unexpected) failures on different Lisps. The results may be found here: http://common-lisp.net/project/cl-test-grid/pivot_ql-lib_lisp.html Clicking the ok/fail status refer to the test logs with the failures details. Best regards, - Anton

4 7

Re: [cffi-devel] 64bit hosts without multilib
by Luís Oliveira 06 Jan '12

06 Jan '12

On Wed, Jan 4, 2012 at 12:51 AM, Elias Pipping <pipping(a)lavabit.com> wrote: > Checking for /lib/ld-linux.so.2 would be one possibility. That would only take care of Linux. I'm afraid my Makefile-fu is weak and I don't see an obvious way to fix this. I've registered this bug here: <https://bugs.launchpad.net/cffi/+bug/911557>. Cheers, -- Luís Oliveira http://r42.eu/~luis/

3 3

Re: [cffi-devel] CFFI and Allegro common lisp
by Luís Oliveira 05 Jan '12

05 Jan '12

Hello John, git blame says you sent us a patch to add long-long on Allegro on 64-bit platforms. Robert says that isn't quite working. Do you have any recollection of any limitations of this long-long support or something like that? Meanwhile, I'll revert the patch: <https://github.com/cffi/cffi/commit/80a06643361af383cadcbb7232ba3d73b828a96d> unless you have some objection since I don't have access to a 64-bit ACL. On Thu, Jan 5, 2012 at 8:18 PM, Robert Goldman <rpgoldman(a)sift.info> wrote: > I am pretty certain that *no* version of Allegro supports :long :long. > I have verified this on 64-bit ACL for both Mac OS X and Linux. So I > would strongly urge you to record that ACL has no-long-long. Cheers, -- Luís Oliveira http://r42.eu/~luis/

3 3