* The former definition for :long-long types in cffi-ecl.lisp was broken. I have agumented ECL with a feature that signals the existence of such a type in ECL and include a patch here for CFFI to take that into account. (Patch attached)
* Instead of defining NULL-POINTER-P, it would be better to simply reexport the symbol living in the "EXT" package. Patch attached.
* Upon reading the CFFI specification it seems that FOREIGN-FREE can only free memory that has been allocated by CFFI. However the test cases in misc-types.lsp do something else, deallocating the output of my_strdup() explicitely.
* At the low level ECL has two different foreign function interfaces: one used in the interpreter and relying on an external library (libffi) and another one, much simpler, using the C compiler. Right now CFFI was only using the former unless it was not available. I provide a patch that chooses the interface depending on the use of the code: interpreter or compiled. (patch attached).
* Is there the equivalent of launchpad for CFFI? Should I always submit the patches to the mailing list? I say this because there are other improvements I could forward when I find time.
Cheers
Juanjo
On Sun, Apr 25, 2010 at 1:47 PM, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
- The former definition for :long-long types in cffi-ecl.lisp was broken. I
have agumented ECL with a feature that signals the existence of such a type in ECL and include a patch here for CFFI to take that into account. (Patch attached)
I've tweaked your patch to use that feature to decide when to push cffi-sys::no-long-long to *features*.
- Instead of defining NULL-POINTER-P, it would be better to simply reexport
the symbol living in the "EXT" package. Patch attached.
Pushed.
- Upon reading the CFFI specification it seems that FOREIGN-FREE can only
free memory that has been allocated by CFFI. However the test cases in misc-types.lsp do something else, deallocating the output of my_strdup() explicitely.
That was the probably the original intent. Is this problematic for ECL? (I'm guessing it's because of the GC?)
- At the low level ECL has two different foreign function interfaces: one
used in the interpreter and relying on an external library (libffi) and another one, much simpler, using the C compiler. Right now CFFI was only using the former unless it was not available. I provide a patch that chooses the interface depending on the use of the code: interpreter or compiled. (patch attached).
Cool, pushed.
- Is there the equivalent of launchpad for CFFI? Should I always submit the
patches to the mailing list? I say this because there are other improvements I could forward when I find time.
http://launchpad.net/cffi exists. I've been slowly populating it with my TODO list. The mailing list is probably the best venue for patches for now, though.
On Mon, Apr 26, 2010 at 12:57 AM, Luís Oliveira luismbo@gmail.com wrote:
On Sun, Apr 25, 2010 at 1:47 PM, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
- The former definition for :long-long types in cffi-ecl.lisp was broken.
I
have agumented ECL with a feature that signals the existence of such a
type
in ECL and include a patch here for CFFI to take that into account.
(Patch
attached)
I've tweaked your patch to use that feature to decide when to push cffi-sys::no-long-long to *features*.
Please don't. LONG-LONG is not supported by the interpreter backend (libffi) because it considers it a complex type. Until I figure out how to use it, we should still stick with that.
- Upon reading the CFFI specification it seems that FOREIGN-FREE can only
free memory that has been allocated by CFFI. However the test cases in misc-types.lsp do something else, deallocating the output of my_strdup() explicitely.
That was the probably the original intent. Is this problematic for ECL? (I'm guessing it's because of the GC?)
It is very problematic because ECL allocates and frees foreign memory using the garbage collector. If the user applies FOREIGN-FREE upon a pointer that was NOT created using CFFI, then ECL's garbage collector will choke. The tests we causing ECL to hang.
Juanjo
On Mon, Apr 26, 2010 at 9:07 AM, Juan Jose Garcia-Ripoll < juanjose.garciaripoll@googlemail.com> wrote:
That was the probably the original intent. Is this problematic for
ECL? (I'm guessing it's because of the GC?)
It is very problematic because ECL allocates and frees foreign memory using the garbage collector. If the user applies FOREIGN-FREE upon a pointer that was NOT created using CFFI, then ECL's garbage collector will choke. The tests we causing ECL to hang.
I attach you an updated patch that fixes CFFI's test suite. Could you please include it in the tree? With the current abuse of FOREIGN-FREE ECL hangs and this is causing my test farm a lot of headaches.
Juanjo
On Mon, Apr 26, 2010 at 1:51 PM, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
It is very problematic because ECL allocates and frees foreign memory using the garbage collector. If the user applies FOREIGN-FREE upon a pointer that was NOT created using CFFI, then ECL's garbage collector will choke. The tests we causing ECL to hang.
I see. What's the rationale? Could ECL's FOREIGN-FREE/ALLOC use plain malloc()/free()? If not, CFFI should probably provide something like LIBC-MALLOC and LIBC-FREE explicitly.
I attach you an updated patch that fixes CFFI's test suite. Could you please include it in the tree? With the current abuse of FOREIGN-FREE ECL hangs and this is causing my test farm a lot of headaches.
Pushed, as a temporary solution.
Thanks,
On Thu, Apr 29, 2010 at 1:02 AM, Luís Oliveira luismbo@gmail.com wrote:
On Mon, Apr 26, 2010 at 1:51 PM, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
It is very problematic because ECL allocates and frees foreign memory using the garbage collector. If the user applies FOREIGN-FREE upon a pointer that was NOT created using CFFI, then ECL's garbage collector
will
choke. The tests we causing ECL to hang.
I see. What's the rationale? Could ECL's FOREIGN-FREE/ALLOC use plain malloc()/free()?
But why? As we said there is nothing in the CFFI specification that mandates use of malloc(), just that the memory will not be freed http://common-lisp.net/project/cffi/manual/html_node/foreign_002dalloc.html
Current system uses the same foreign memory manager as ECL, because it is safer for us, allows a good coexistence of both sets of memory and allows us to keep track of the amount of memory used. Furthermore, should it be needed, some pointers can be allocated as collectable and automatically reclaimed by our GC -- not that it is done by default, though --. This has worked fine, it was the way UFFI worked, and I see no reason to change.
Also, except for OpenMCL I see no explicit use of malloc/free anywhere. It must be some kind of accident that it works with other implementations.
Juanjo
On Thu, Apr 29, 2010 at 8:49 AM, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
I see. What's the rationale? Could ECL's FOREIGN-FREE/ALLOC use plain malloc()/free()?
But why? As we said there is nothing in the CFFI specification that mandates use of malloc(), just that the memory will not be freed
Right. The CFFI specification/documentation is a work in progress though. I believe one of UFFI's major shortcomings is that it fails to ensure consistent behaviour across implementations for error situations. CFFI is much better in that regard, but it certainly could use more work. The issue of whether FOREIGN-FREE/ALLOC should be implemented in terms of malloc()/free() is one such example.
Current system uses the same foreign memory manager as ECL, because it is safer for us, allows a good coexistence of both sets of memory and allows us to keep track of the amount of memory used. Furthermore, should it be needed, some pointers can be allocated as collectable and automatically reclaimed by our GC -- not that it is done by default, though
I'm convinced. :-) I'll update the manual to be explicit about this. Using DEFCFUN to get at malloc()/free() should be straightforward anyway, except we don't have a size_t type.
Also, except for OpenMCL I see no explicit use of malloc/free anywhere. It must be some kind of accident that it works with other implementations.
SBCL calls malloc/free under the hood, as do most other implementations. I think Allegro uses a different allocator.
-
Juan Jose Garcia-Ripoll -
Luís Oliveira