New failure in cl-libuv I don't understand
Full log is here: http://report.quicklisp.org/2023-05-29/failure-report/cl-libuv.html#cl-libuv Here's part of it: ; file: /home/quicklisp/quicklisp-controller/dist/build-cache/cl-libuv/9f0aced1c6a2ef1737696a9a577f65fbf922334a/cl-libuv-20230529-git/bindings.lisp ; in: DEFCFUN ("uv_loop_size" UV-LOOP-SIZE) ; (CFFI:DEFCFUN ("uv_loop_size" LIBUV::UV-LOOP-SIZE) ; LIBUV::SIZE-T) ; ; caught ERROR: ; (during macroexpansion of (DEFCFUN ("uv_loop_size" UV-LOOP-SIZE) ...)) ; Unknown CFFI type SIZE-T If I try compiling the library in isolation, it seems to work. Is it possible I've got some kind of stale fasl environment going on? Is there more research I can do to understand why this happens? Thanks, Zach
cffi-grovel processes cl-libuv's grovel.lisp, produces a C file, compiles it with cc, runs the resulting executable to produce a Lisp file which is then compiled and loaded. That Lisp file is what contains a defctype form that defines the libuv::size-t type which is then referenced by the next file in cl-libuv.asd. The ASDF integration bits for the cffi-grovel component are in cffi/grovel/asdf.lisp. Maybe some tracing and/or print debugging there can provide a clue. HTH, Luís On Tue, 30 May 2023, 07:48 Zach Beane, <xach@xach.com> wrote:
Full log is here:
http://report.quicklisp.org/2023-05-29/failure-report/cl-libuv.html#cl-libuv
Here's part of it:
; file: /home/quicklisp/quicklisp-controller/dist/build-cache/cl-libuv/9f0aced1c6a2ef1737696a9a577f65fbf922334a/cl-libuv-20230529-git/bindings.lisp ; in: DEFCFUN ("uv_loop_size" UV-LOOP-SIZE) ; (CFFI:DEFCFUN ("uv_loop_size" LIBUV::UV-LOOP-SIZE) ; LIBUV::SIZE-T) ; ; caught ERROR: ; (during macroexpansion of (DEFCFUN ("uv_loop_size" UV-LOOP-SIZE) ...)) ; Unknown CFFI type SIZE-T
If I try compiling the library in isolation, it seems to work. Is it possible I've got some kind of stale fasl environment going on? Is there more research I can do to understand why this happens?
Thanks, Zach
If I try compiling the library in isolation, it seems to work.
in isolation, but in the same environment? a quick idea: maybe a newer/stricter GCC fails to produce the executable, or the C definition got relocated into another .h file... then the CFFI ASDF goveler integration is too fail tolerant, and instead of failing, it just prints some error higher up in the log that you haven't noticed? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Exploitation is a word often used but rarely defined. In its most literal meaning — I 'exploit' you if I in some way benefit from your existence — it is the reason human society exists. We all benefit from one another's existence. We all exploit each other.” — David D. Friedman (1945–), 'The Machinery of Freedom' (1973)
On Tue, 30 May 2023 at 10:13, Attila Lendvai <attila.lendvai@gmail.com> wrote:
a quick idea: maybe a newer/stricter GCC fails to produce the executable, or the C definition got relocated into another .h file...
Perhaps cl-libuv's grovel.lisp should explicitly (include "stddef.h")? 🤷♂️ -- Luís Oliveira https://kerno.org/~luis/
* Luis Oliveira <CAB-HnLRJnFz5Kd5CiybQepgGnUmgMw=OYHq4rsd-XF8Nnbf8XQ@mail.gmail.com> Wrote on Tue, 30 May 2023 11:33:05 +0100
On Tue, 30 May 2023 at 10:13, Attila Lendvai <attila.lendvai@gmail.com> wrote:
a quick idea: maybe a newer/stricter GCC fails to produce the executable, or the C definition got relocated into another .h file...
Perhaps cl-libuv's grovel.lisp should explicitly (include "stddef.h")?
No, the problem is in libuv commit dd6df070e0e34 on 2023-05-23 21:56:08 which introduces a form at the top of the file grovel.lisp: ``` (in-package :libuv) +#.(when (uiop:getenv "HOMEBREW_PREFIX") + (pushnew :homebrew *features*) + (values)) + ``` This fools the loop in grovel/grove.lisp (generate-c-file) which has ``` (flet ((read-forms (s) (do ((forms ()) (form (read s nil nil) (read s nil nil))) ((null form) (nreverse forms)) ``` The first form is "(in-package :libuv)" If your environment doesn't have HOMEBREW_PREFIX the second form is read with READ-EVAL T as NIL and the reader loop exits and you end up with an empty C file with nothing other than the inpackage form. You could change the loop, but I'd suggest that you don't use top-level reader conditionals in the grovel.lisp file, set up the features outside of the grovel.lisp file. ---
FTR, Madhu's analysis has been forwarded to the author: https://github.com/orthecreedence/cl-libuv/issues/23 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Nobody in the world, nobody in history, has ever gotten their freedom by appealing to the moral sense of the people who were oppressing them.” — Assata Shakur (1947–)
This one was my fault. I did the PR with this patch, because cl-libuv wouldn't build on my Mac. Since there's no testing, apparently, it slipped through. I don't know how to fix this, because I have never used CFFI-grovel, so no idea how to put Madhu's advice into practice. As an aside, doesn't this kind of break Faré's design principle that the person who has the information is the one who should be configuring the software? It seems like we are expecting the programmer to guess ahead of time what are all of the possible locations for the foreign library and includes. Wouldn't it make more sense for this to be configurable? Of course, this is partly my fault, since ASDF does not support configuring operations, except with gross dynamic variables. On 8 Jun 2023, at 15:32, Attila Lendvai wrote:
FTR, Madhu's analysis has been forwarded to the author:
https://github.com/orthecreedence/cl-libuv/issues/23
-- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Nobody in the world, nobody in history, has ever gotten their freedom by appealing to the moral sense of the people who were oppressing them.” — Assata Shakur (1947–)
Hi Robert,
This one was my fault. I did the PR with this patch, because cl-libuv wouldn't build on my Mac. Since there's no testing, apparently, it slipped through.
I don't know how to fix this, because I have never used CFFI-grovel, so no idea how to put Madhu's advice into practice.
As an aside, doesn't this kind of break Faré's design principle that the person who has the information is the one who should be configuring the software? It seems like we are expecting the programmer to guess ahead of time what are all of the possible locations for the foreign library and includes. Wouldn't it make more sense for this to be configurable?
Yes, but there's no way to currently configure this except as you point out below.
Of course, this is partly my fault, since ASDF does not support configuring operations, except with gross dynamic variables.
Adding a configuration stage for a DEFSYSTEM would be my most desired feature at the moment. -- Stelian Ionescu
* "Stelian Ionescu" <38aea509-0fe1-4894-b7f7-579275c70229@app.fastmail.com> Wrote on Thu, 08 Jun 2023 21:59:16 -0400 [RPG]
This one was my fault. I did the PR with this patch, because cl-libuv wouldn't build on my Mac. Since there's no testing, apparently, it slipped through.
I don't know how to fix this, because I have never used CFFI-grovel, so no idea how to put Madhu's advice into practice.
My suggestion was to move the call which did ``` #.(when (uiop:getenv "HOMEBREW_PREFIX") (pushnew :homebrew *features*) (values)) ``` out of grovel.lisp, and set up the features before the groveler is called. Note this could be done *anywhere* except in grovel.lisp, as long as it is done before grovel.lisp is processed. And there is no need it to be #. macro. I don't understand what problem you percieve.
As an aside, doesn't this kind of break Faré's design principle that the person who has the information is the one who should be configuring the software? It seems like we are expecting the programmer to guess ahead of time what are all of the possible locations for the foreign library and includes. Wouldn't it make more sense for this to be configurable?
Yes, but there's no way to currently configure this except as you point out below.
Of course, this is partly my fault, since ASDF does not support configuring operations, except with gross dynamic variables.
Adding a configuration stage for a DEFSYSTEM would be my most desired feature at the moment.
What? There is no need to complicate defsystem anymore. you just define a system called libuv-config which loads a designated configuration file and make your libuv system depend on libuv-config which gets loaded before it. If this incident teaches it is that the mechanisms to do it are already there, you just need to make correct use of it. Not use an incorrect usage of existing mechanisms as an excuse to add new backdoor APIs
[...]
What? There is no need to complicate defsystem anymore. you just define a system called libuv-config which loads a designated configuration file and make your libuv system depend on libuv-config which gets loaded before it.
There is a great need for a way to scope *features* to a DEFSYSTEM so that they don't bleed into the global environment, and for features to be not a set but key-value pairs. -- Stelian Ionescu
On 8 Jun 2023, at 21:34, Madhu wrote:
* "Stelian Ionescu" <38aea509-0fe1-4894-b7f7-579275c70229@app.fastmail.com> Wrote on Thu, 08 Jun 2023 21:59:16 -0400 [RPG]
This one was my fault. I did the PR with this patch, because cl-libuv wouldn't build on my Mac. Since there's no testing, apparently, it slipped through.
I don't know how to fix this, because I have never used CFFI-grovel, so no idea how to put Madhu's advice into practice.
My suggestion was to move the call which did
``` #.(when (uiop:getenv "HOMEBREW_PREFIX") (pushnew :homebrew *features*) (values)) ```
out of grovel.lisp, and set up the features before the groveler is called. Note this could be done *anywhere* except in grovel.lisp, as long as it is done before grovel.lisp is processed. And there is no need it to be #. macro.
I don't understand what problem you percieve.
It's simple: I don't understand cl-libuv well enough to understand what location to move this to. What I did was my best guess. I encourage you to submit a PR or an issue with a proposed move destination to cl-libuv.
As an aside, doesn't this kind of break Faré's design principle that the person who has the information is the one who should be configuring the software? It seems like we are expecting the programmer to guess ahead of time what are all of the possible locations for the foreign library and includes. Wouldn't it make more sense for this to be configurable?
Yes, but there's no way to currently configure this except as you point out below.
Of course, this is partly my fault, since ASDF does not support configuring operations, except with gross dynamic variables.
Adding a configuration stage for a DEFSYSTEM would be my most desired feature at the moment.
What? There is no need to complicate defsystem anymore. you just define a system called libuv-config which loads a designated configuration file and make your libuv system depend on libuv-config which gets loaded before it.
Probably the easiest thing is to read `LIBUV_LOCATION` and `LIBUV_HEADER_LOCATION` environment variables if they are present.
Another possible fix is to always return zero values, e.g. #.(progn (when (uiop:getenv "HOMEBREW_PREFIX") (pushnew :homebrew *features*)) (values)) __Martin
On Thu, 08 Jun 2023 16:32:36 -0500, Robert Goldman said:
This one was my fault. I did the PR with this patch, because cl-libuv wouldn't build on my Mac. Since there's no testing, apparently, it slipped through.
I don't know how to fix this, because I have never used CFFI-grovel, so no idea how to put Madhu's advice into practice.
As an aside, doesn't this kind of break Faré's design principle that the person who has the information is the one who should be configuring the software? It seems like we are expecting the programmer to guess ahead of time what are all of the possible locations for the foreign library and includes. Wouldn't it make more sense for this to be configurable?
Of course, this is partly my fault, since ASDF does not support configuring operations, except with gross dynamic variables.
On 8 Jun 2023, at 15:32, Attila Lendvai wrote:
FTR, Madhu's analysis has been forwarded to the author:
https://github.com/orthecreedence/cl-libuv/issues/23
-- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Nobody in the world, nobody in history, has ever gotten their freedom by appealing to the moral sense of the people who were oppressing them.” — Assata Shakur (1947–)
On Fri, 9 Jun 2023 at 14:24, Martin Simmons <martin@lispworks.com> wrote:
Another possible fix is to always return zero values, e.g.
#.(progn (when (uiop:getenv "HOMEBREW_PREFIX") (pushnew :homebrew *features*)) (values))
Heh. This is a lovely hack, but I'd accept a patch fixing the READ loop that Madhu identified to test for EOF rather than NIL. Also, we could add a :PROGN directive to cffi-grovel for arbitrary code like this. -- Luís Oliveira https://kerno.org/~luis/
participants (7)
-
Attila Lendvai -
Luís Oliveira -
Madhu -
Martin Simmons -
Robert Goldman -
Stelian Ionescu -
Zach Beane