Current ML is ecl-devel@common-lisp.net, I'm forwarding this mail there.
Daniel Kochmański writes:
Faré writes:
[Adding ECL-list to this conversation, that I believe should be publicly archived]
Good idea.
On Fri, Oct 2, 2015 at 7:24 AM, Daniel Kochmański daniel@turtleware.eu wrote:
Hey,
when fixing problem with randomized names etc I have taken a closer look at the build process. You asked earlier how ECL manages to do monolithic builds.
Each module is initialized with a special function which registers the whole module and creates lisp objects. Before calling this function ECL isn't aware of the existance of it neither it has environment.
Module may be composed on other modules. Builder, when constructing initialization function for "top module" gathers all submodules (object files or libraries) and parses the binary file to find special tag which contains name of functions which initialize submodules. Top module in it's initialization functions calls (beside the other things) all submodule initialization functions. Then it concatanates all of them in .a, .fasl, .so, ... files
Top-module Sub-module-1 Sub-module-2 …
So it is guaranteed, that if this top-module will be embedded in the other one it's special tag will be found first and the correct initialization function will be called (which will initialize the futher submodules).
This description might be not 100% accurate but I believe it is how things work there. Reverse engineering might be misleading :-)
I've fixed bundles once again and ASDF passes tests. It's your call if you want to enable it by default.
Best regards, Daniel
-- Daniel Kochmański | Poznań, Poland ;; aka jackdaniel
"Be the change that you wish to see in the world." - Mahatma Gandhi
Thanks for these explanations.
Questions:
1- what is a "tag" in this context?
Tag is a string embedded in the C code with deterministic prefix followed by a name of the object function. For instance assuming initialization function has name lib_init_xyz, then it will look something like:
void lib_init_xyz (cl_object block) { ... Cblock->cblock.data_text = (const cl_object *)"@EcLtAg_lib:lib_init_xyz@"; ... }
When emiting code for top module we scan the binary object files for appropriate tag (in this example @EcLAg_lib) to find the inititalization function name which we embed in a source code as a normall call:
void lib_init_top_module (cl_object block) { ... Cblock->cblock.data_text = (const cl_object *)"@EcLtAg_lib:lib_init_top_module@"; ... lib_init_xyz(block); ... }
Note that now function names are randomized and this is mere a psudocode. For further details consult src/cmp/cmpmain.lsp.
2- once again, why not use the linker support for initialization? If behavior has to depend on whether ECL was initialized yet, the initialization could consider in calling ecl_register_init_function, which depending on whether ECL was booted, would either call the function and/or add it to a hook. See once again attached files on how to use linker support for initialization.
Fact that we might bundle many objects in one archive doesn't mean that they don't have dependencies on one another. *I think* that we have to strictly control initialization order. It's possible that I'm wrong here though.
Also do we want to *always* initialize *everything* what is linked? We may have compiled-in support for number of lisp libraries in one module just for conveniance and require them on demand depending on the application using our code.
I'm guessing here since it's a design decision made by someone else. Your proposition is surely worth investigating - it is like it is because it was that way when I approached the codebase.
3- How does MKCL do it?
MKCL creates these initialization functions but when it loads files it "slurps" some vector with objects - it doesn't call initialization functions. Don't know if function creation is rendundant/obsolete/inconsistent/correct - you have to ask MKCL maintainer for details. It looks to me that it works for files built with MKCL but breaks linking from the outer code with the libraries created by it (and by breakage I mean that compilation goes fine but such modules doesn't work). Once again I'm not 100% about that. MKCL code diverged a lot with this regards.
Regards, Daniel
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Suppose 2/3 Congress were incinerated, what would we lose to offset our gain of their salaries and the salaries of their parasites? — H. L. Mencken