unwanted recompilation with :CFFI-GROVEL-FILE
Hello, Sorry if this is the wrong place to ask ... I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ <https://common-lisp.net/project/osicat/> and https://github.com/osicat/osicat <https://github.com/osicat/osicat> ) in SBCL on OSX. whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the cffi-grovel-files and their dependencies. Is this normal, or is there some way to make it behave like asdf usually does, that it recompiles only when a file has changed? I know that this issue has 3 components (asdf, cffi, and osicat), but I thought I’d first check whether this is an unavoidable feature of using grovel files. If not, I can try to fix osicat. Here are some of the compilation messages: https://pastebin.com/GKArFCWc <https://pastebin.com/GKArFCWc> JK
A followup: this behavior stops when I comment out (:cffi-wrapper-file "wrappers" :soname "libosicat”) in osicat’s asd file. I understand that wrappers are to generate functions from C macros, necessitating a C compilation step to build a dynamic library that is then loaded. For example in libosicat, the following line appears to turn a C macro into a function (defwrapper* "s_isreg" ("int" :boolean) ((mode ("mode_t" mode))) "return S_ISREG(mode);”) Apparently wrappers.lisp becomes wrappers.processed-wrapper-file, which, despite not changing upon reload of osicat system, is recompiled anyway, resulting in a new wrappers.fasl. The dynamic library resulting from this (libosicat.dylib on OSX) is not rebuilt, however. In short, :CFFI-WRAPPER-FILE seems to trigger unnecessary recompilation of an intermediate lisp file processed from the original lisp source, but not creation of this intermediate file, nor the recompilation of the dynamic library. Using the latest git version of CFFI doesn’t seem to help.
On Apr 14, 2020, at 1:06 PM, JTK <jetmonk@gmail.com> wrote:
Hello,
Sorry if this is the wrong place to ask ...
I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ <https://common-lisp.net/project/osicat/> and https://github.com/osicat/osicat <https://github.com/osicat/osicat> ) in SBCL on OSX.
whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the cffi-grovel-files and their dependencies.
Is this normal, or is there some way to make it behave like asdf usually does, that it recompiles only when a file has changed?
I know that this issue has 3 components (asdf, cffi, and osicat), but I thought I’d first check whether this is an unavoidable feature of using grovel files. If not, I can try to fix osicat.
Here are some of the compilation messages: https://pastebin.com/GKArFCWc <https://pastebin.com/GKArFCWc>
JK
Sorry for all these emails, but this trivial case reproduces the recompilation, so it isn’t osicat specific. https://github.com/jetmonk/test-grovel <https://github.com/jetmonk/test-grovel>
On Apr 14, 2020, at 4:31 PM, JTK <jetmonk@gmail.com> wrote:
A followup: this behavior stops when I comment out (:cffi-wrapper-file "wrappers" :soname "libosicat”) in osicat’s asd file.
I understand that wrappers are to generate functions from C macros, necessitating a C compilation step to build a dynamic library that is then loaded.
For example in libosicat, the following line appears to turn a C macro into a function
(defwrapper* "s_isreg" ("int" :boolean) ((mode ("mode_t" mode))) "return S_ISREG(mode);”)
Apparently wrappers.lisp becomes wrappers.processed-wrapper-file, which, despite not changing upon reload of osicat system, is recompiled anyway, resulting in a new wrappers.fasl.
The dynamic library resulting from this (libosicat.dylib on OSX) is not rebuilt, however.
In short, :CFFI-WRAPPER-FILE seems to trigger unnecessary recompilation of an intermediate lisp file processed from the original lisp source, but not creation of this intermediate file, nor the recompilation of the dynamic library.
Using the latest git version of CFFI doesn’t seem to help.
On Apr 14, 2020, at 1:06 PM, JTK <jetmonk@gmail.com <mailto:jetmonk@gmail.com>> wrote:
Hello,
Sorry if this is the wrong place to ask ...
I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ <https://common-lisp.net/project/osicat/> and https://github.com/osicat/osicat <https://github.com/osicat/osicat> ) in SBCL on OSX.
whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the cffi-grovel-files and their dependencies.
Is this normal, or is there some way to make it behave like asdf usually does, that it recompiles only when a file has changed?
I know that this issue has 3 components (asdf, cffi, and osicat), but I thought I’d first check whether this is an unavoidable feature of using grovel files. If not, I can try to fix osicat.
Here are some of the compilation messages: https://pastebin.com/GKArFCWc <https://pastebin.com/GKArFCWc>
JK
Hello,
Sorry if this is the wrong place to ask ...
I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ and https://github.com/osicat/osicat ) in SBCL on OSX.
whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the cffi-grovel-files and their dependencies.
Is this normal, or is there some way to make it behave like asdf usually does, that it recompiles only when a file has changed?
This sounds very much like a bug in ASDF that was fixed a while ago. I suggest upgrading to the latest version and retrying. -- Stelian Ionescu
Thank you for the suggestion to upgrade ASDF, Stelian. I upgraded my builtin SBCL asdf.fasl from 3.3.4 to 3.3.4.1 (git version) with a manual replacement of fasl. This - fixes my toy test case on OSX+SBCL (no more unnecessary recompile) - does not fix osicat on OSX+SBCL - but both osicat and toy test case work in ccl64+OSX with no ASDF fixes needed. - but osicat works without reloading in SBCL+Linux64, again without an ASDF upgrade. So it looks like it could be some kind of glitch in asdf or OSX SBCL (up to 1.4.9). JK
On Apr 14, 2020, at 5:40 PM, Stelian Ionescu <sionescu@cddr.org> wrote:
Hello,
Sorry if this is the wrong place to ask ...
I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ <https://common-lisp.net/project/osicat/> and https://github.com/osicat/osicat <https://github.com/osicat/osicat> ) in SBCL on OSX.
whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the cffi-grovel-files and their dependencies.
Is this normal, or is there some way to make it behave like asdf usually does, that it recompiles only when a file has changed?
This sounds very much like a bug in ASDF that was fixed a while ago. I suggest upgrading to the latest version and retrying.
-- Stelian Ionescu
It turns out to be an intermediate-version SBCL bug, fixed by upgrading to 2.0.3. Sorry for this additional email, but I thought it better to close the issue out.
On Apr 15, 2020, at 12:36 PM, JTK <jetmonk@gmail.com> wrote:
Thank you for the suggestion to upgrade ASDF, Stelian.
I upgraded my builtin SBCL asdf.fasl from 3.3.4 to 3.3.4.1 (git version) with a manual replacement of fasl.
This
- fixes my toy test case on OSX+SBCL (no more unnecessary recompile)
- does not fix osicat on OSX+SBCL
- but both osicat and toy test case work in ccl64+OSX with no ASDF fixes needed.
- but osicat works without reloading in SBCL+Linux64, again without an ASDF upgrade.
So it looks like it could be some kind of glitch in asdf or OSX SBCL (up to 1.4.9).
JK
On Apr 14, 2020, at 5:40 PM, Stelian Ionescu <sionescu@cddr.org <mailto:sionescu@cddr.org>> wrote:
Hello,
Sorry if this is the wrong place to ask ...
I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ <https://common-lisp.net/project/osicat/> and https://github.com/osicat/osicat <https://github.com/osicat/osicat> ) in SBCL on OSX.
whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the cffi-grovel-files and their dependencies.
Is this normal, or is there some way to make it behave like asdf usually does, that it recompiles only when a file has changed?
This sounds very much like a bug in ASDF that was fixed a while ago. I suggest upgrading to the latest version and retrying.
-- Stelian Ionescu
participants (2)
-
JTK -
Stelian Ionescu