Hi folks,
I'm currently working on adding caching of .so and wrapper files to cffi. This is to handle issues arising from trying to use various libraries on machines without a C compiler. I'm hoping you can take a peek at my branch [0] and see if I'm going in the right direction.
The changes are very simple, we allow users to specify a cache directory by using an optional :cache-dir argument in :grovel-file & :wrapper-file forms. CFFI will then look there first for the required files before trying to compile them itself.
The tricky bit is that we recommend that people use the #+ & #- reader conditionals in the grovel and wrapper specifications, which means the cached results are only applicable if all the required feature-forms match.
To handle this I am currently using my with-cached-reader-conditionals library [1] which, whilst small and standalone, could be replicated inside CFFI if required. The library modifies the readtable so that feature expression still work as before, but the also record the feature-forms used.
With the captured feature information we can make a subdirectory inside the :cache-dir directory, whose name is based on the feature information. In my current experiment it does something simple & easy to read, however it would likely make directory names too long for windows users.
For example I took osicat and added a cache directory to the following lines:
(:grovel-file "basic-unixint" :cache-dir "foo") (:wrapper-file "wrappers" :soname "libosicat" :cache-dir "foo")
The "foo" directory has the following contents
bsd_nil_darwin_nil_freebsd_nil_linux_t_openbsd_nil unixint.processed-grovel-file darwin_nil_linux_t_mips_nil_openbsd_nil_windows_nil basic-unixint.processed-grovel-file linux_t_windows_nil libosicat.so wrappers.processed-wrapper-file
All the system does is copy these files to the same directory the system would have put the compiled results in the standard system.
My next addition is going to be a function the user can call that will do the following:
- Work out if cffi-grovel has cached files for the grovel and wrapper results - If it didnt have those files it will copy the cachable files to a user specified location (or maybe just the cache directory)
The idea of this is that it will allow the users to easily dontate their cache files to the project which hopefully means we can better coverage quickly
Thoughts would be very welcome, I hope this finds you all well,
Baggers
[0] https://github.com/cbaggers/cffi/tree/feature-grovel-caching [1] https://github.com/cbaggers/with-cached-reader-conditionals