Re: Statically linking C code into ECL
Faré writes:
What's the official way to link a C file into ECL?
What do you mean by 'linking C file'? If you want to compile independent C file with ECL, then you are out of luck, because there is no official support for that in ECL's builder. It may work by passing file as one of lisp-files to builder, because ECL uses C files *internally*, but it's far from encouraged or official. As a side note, I see you use there c::builder, which is also internal. Exported interfaces are build-fasl, build-program, build-static-library and build-shared-library (it doesn't make a difference right now, but in the future builder interface may change, while the mentioned functions wont).
I'm adding a :c-file type to cffi-toolchain, and it works great on sbcl and clisp, or on ECL when loading the file as a .so, but when I try to link the file into an executable, the .o failed to be included in the executable: the .o was successfully grouped with other .o's from the same system in a .a, but ecl fails to use -Wl,--whole-archive, so the .o is not preserved. Is there a way to tell ECL to use -Wl,--whole-archive and/or (on Darwin) -force_load before every .a?
builder (and all interfaces built on top of it) support ld-flags key argument. Additionally if you want to influence how ECL compiles files dynamically you may shadow *user-cc-flags* and *user-ld-flags*.
The question is in the context of my cffi branch (that I'm hoping to upstream asap, pending merge of the sbcl patch it depends on): https://github.com/fare/cffi
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org A president worth voting for wouldn't run for office.
If this remarks aren't helpful, please tell me exactly what do you want to achieve with some minimal example (i.e I'm doing that and that, expecting this and this, but it doesn't work failing in this way), because I don't quite understand your problem. I'm adding ecl-devel to CC, because someone may be more competent to answer your question. Regards, Daniel -- Daniel Kochmański ;; aka jackdaniel | Przemyśl, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
On Mon, Dec 26, 2016 at 5:02 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
What do you mean by 'linking C file'? If you want to compile independent C file with ECL, then you are out of luck, because there is no official support for that in ECL's builder. It may work by passing file as one of lisp-files to builder, because ECL uses C files *internally*, but it's far from encouraged or official.
Is there an official or encouraged way? I'm trying to achieve for ECL what I now have for SBCL and CLISP, a single-executable output with both Lisp and C code linked together. If I link things myself, I don't have the magic ecl initialization code; if I don't, I can't convince ECL to pass the correct flags.
As a side note, I see you use there c::builder, which is also internal. Exported interfaces are build-fasl, build-program, build-static-library and build-shared-library (it doesn't make a difference right now, but in the future builder interface may change, while the mentioned functions wont).
I was "just" reusing the old pattern from asdf-ecl, and hadn't noticed the new interface. I will update cffi-toolchain and uiop/image.
builder (and all interfaces built on top of it) support ld-flags key argument. Additionally if you want to influence how ECL compiles files dynamically you may shadow *user-cc-flags* and *user-ld-flags*.
This won't work. *user-cc-flags* isn't used by build-program, AFAICT, and user-ld-flags is only used at the end of the linking command, which is too late for a --whole-archive (and a --no-whole-archive is actually needed before the -ldl -lm etc. calls, whereas on Darwin, the -force_load is needed in front of each individual .a). I suppose I could force linking of .o's and never .a's. Sigh. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Mathematics is as little a science as grammar is a language. — Ernst Mayr
Faré writes:
Is there an official or encouraged way?
No.
builder (and all interfaces built on top of it) support ld-flags key argument. Additionally if you want to influence how ECL compiles files dynamically you may shadow *user-cc-flags* and *user-ld-flags*.
This won't work. *user-cc-flags* isn't used by build-program, AFAICT, and user-ld-flags is only used at the end of the linking command, which is too late for a --whole-archive (and a --no-whole-archive is actually needed before the -ldl -lm etc. calls, whereas on Darwin, the -force_load is needed in front of each individual .a).
*user-cc-flags* and *user-ld-flags* are used in all operations involving C/C++ compiler. If it's too late, then what about (mentioned in the previous mail) :ld-flags ? (let ((c::*compile-verbose* t) (*compile-print* t)) (c::builder :program "sdbam" :lisp-files '("/home/jack/test.c") :ld-flags '("ld-flag-bam=3"))) (RUN-PROGRAM "gcc" ("-o" "sdbam" "-L/usr/local/lib/" "/tmp/eclinitPjB9n2.o" "test.c" "ld-flag-bam=3" "-Wl,--rpath,/usr/local/lib/" "-lecl" "-lgmp" "-lgc" "-lgc" "-lgc" "-lpthread" "-ldl" "-lm")) ld-flag-bam=3 is passed before -ldl -lm etc (that, of course, won't work, because ld-flag=bam=3 doesn't make much sense), so if that's what you want, just pass :ld-flags to c:build-*. -- Daniel Kochmański ;; aka jackdaniel | Przemyśl, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
On Mon, Dec 26, 2016 at 8:06 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Faré writes:
Is there an official or encouraged way?
No.
Should there be one? I've lost (for a year or two) ability to build OpenAxiom with ECL and I've found it more and more difficult and time consuming to debug building programs with ECL that involves some C and C++ source libraries. PS: I fully understand and appreciate "this is open source, where is your patch" mantra. -- Gaby
Gabriel Dos Reis writes:
On Mon, Dec 26, 2016 at 8:06 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Faré writes:
Is there an official or encouraged way?
No.
Should there be one?
Imho the best way to build C sources against ECL is to produce ELF from ECL (object file, shared object, static library) and use it as one of the libraries you use in your C application (such example is shown in exmaples/build/ – used build tool is make). If you want to use C/C++ libraries in CL application there is FFI for that. To put it in another words, ECL is meant to be embeddedable in C/C++ application, not to make a platform for embedding C/C++ libraries in Common Lisp. The question about an official or encouraged way for compiling lisp files with c files inside the C::BUILDER interface is a question, whenever builder should be used for compiling CL libraries/applications or if it should be a general-purpose building tool (or at least a building tool for both Common Lisp and C/C++). I think that to keep things sane we should stick to the first interpretation, but if there are good reasons for the latter I'm not ruling out adding official support for making C::BUILDER a wrapper around C toolchain. I just don't see valid reasons for that.
I've lost (for a year or two) ability to build OpenAxiom with ECL and I've found it more and more difficult and time consuming to debug building programs with ECL that involves some C and C++ source libraries.
Please file a bug if you can track down the cause of such situation (or provide easy to follow reproduce steps) at https://gitlab.com/embeddable-common-lisp/ecl/issues/ . Best reagards, Daniel -- Daniel Kochmański ;; aka jackdaniel | Przemyśl, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
Unhappily, neither *user-cc-flags* nor *ld-flags* allows me to get the desired result: the former is not used by build-program, and the latter is only used *after* passing the list of .a, too late for a -Wl,--whole-archive. Moreover, on Darwin and/or with lld, I need to use -Wl,-force_load before each individual .a instead, and the current interface won't allow that. I tried to link .o files directly, instead of .a files, but ran into a different ECL issue regarding slot inheritance; see my other email about that. PS: Regarding using c:build-program instead of c::builder :program, when was the change made? Do I need to care about supporting older versions of ECL? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org What is mind? No matter! What is matter? Never mind! — Bertrand Russell's Grand Mother, In Karl Popper, The Unended Quest On Mon, Dec 26, 2016 at 11:06 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Faré writes:
Is there an official or encouraged way?
No.
builder (and all interfaces built on top of it) support ld-flags key argument. Additionally if you want to influence how ECL compiles files dynamically you may shadow *user-cc-flags* and *user-ld-flags*.
This won't work. *user-cc-flags* isn't used by build-program, AFAICT, and user-ld-flags is only used at the end of the linking command, which is too late for a --whole-archive (and a --no-whole-archive is actually needed before the -ldl -lm etc. calls, whereas on Darwin, the -force_load is needed in front of each individual .a).
*user-cc-flags* and *user-ld-flags* are used in all operations involving C/C++ compiler. If it's too late, then what about (mentioned in the previous mail) :ld-flags ?
Faré writes:
Unhappily, neither *user-cc-flags* nor *ld-flags* allows me to get the desired result: the former is not used by build-program, and the latter is only used *after* passing the list of .a, too late for a -Wl,--whole-archive. Moreover, on Darwin and/or with lld, I need to use -Wl,-force_load before each individual .a instead, and the current interface won't allow that.
What about the proposed usage of :ld-flags passed to the builder?
I tried to link .o files directly, instead of .a files, but ran into a different ECL issue regarding slot inheritance; see my other email about that.
Thanks for the report.
PS: Regarding using c:build-program instead of c::builder :program, when was the change made? Do I need to care about supporting older versions of ECL?
It's in place since 2001 (or earlier) according to git blame.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org What is mind? No matter! What is matter? Never mind! — Bertrand Russell's Grand Mother, In Karl Popper, The Unended Quest
Daniel -- Daniel Kochmański ;; aka jackdaniel | Przemyśl, Poland TurtleWare - Daniel Kochmański | www.turtleware.eu "Be the change that you wish to see in the world." - Mahatma Gandhi
On Tue, Dec 27, 2016 at 3:07 AM, Daniel Kochmański <daniel@turtleware.eu> wrote:
Unhappily, neither *user-cc-flags* nor *ld-flags* allows me to get the desired result: the former is not used by build-program, and the latter is only used *after* passing the list of .a, too late for a -Wl,--whole-archive. Moreover, on Darwin and/or with lld, I need to use -Wl,-force_load before each individual .a instead, and the current interface won't allow that.
What about the proposed usage of :ld-flags passed to the builder?
Its behavior is the same as *ld-flags*, and cannot help for the same reasons. I really need that -Wl,--whole-archive before all the .a's and --no-whole-archive after, and/or -force_load flag before each of them on Darwin (or lld).
PS: Regarding using c:build-program instead of c::builder :program, when was the change made? Do I need to care about supporting older versions of ECL?
It's in place since 2001 (or earlier) according to git blame.
Whoa, so even the 2005 code of asdf-ecl was outdated. Well, thanks for that, it will be fixed in 3.2.1 if not 3.2.0. In other news, with a patch to ASDF in !66 and my latest cffi fork, by linking all individual objects rather than .a files, I managed to statically compile an executable with C extension (hello/chello from the bazelisp example): https://gitlab.common-lisp.net/asdf/asdf/merge_requests/66 https://github.com/fare/cffi https://github.com/qitab/bazelisp —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The real danger is that one day machines *will* become intelligent, but we'll lack the mental equipment to notice. — Tiresias, in J.-P. Petit, "Run, Robot, Run"
participants (3)
-
Daniel Kochmański -
Faré -
Gabriel Dos Reis