Dear ECL-developers,
Can one crosscompile ECL for Windows (on a Linux based system)? Currently it fails - both the latest release and the current GIT - with:
$ ./configure --host=i686-w64-mingw32 $ make [...] i686-w64-mingw32-gcc -DECLDIR=""/usr/local/lib/ecl-16.1.2"" -I. -I/tmp/ecl/build -I/tmp/ecl/src/c -I../ecl/gc -DECL_API -DECL_NO_LEGACY -g -O2 -D_THREAD_SAFE -Dmingw32 -c -o threads/process.o threads/process.o.c /tmp/ecl/src/c/threads/process.d:40:22: error: conflicting types for 'GC_CreateThread' extern HANDLE WINAPI GC_CreateThread( ^ In file included from /tmp/ecl/build/ecl/config.h:84:0, from /tmp/ecl/build/ecl/ecl.h:37, from /tmp/ecl/src/c/threads/process.d:20: /tmp/ecl/build/ecl/gc/gc.h:1536:26: note: previous declaration of 'GC_CreateThread' was here GC_API HANDLE WINAPI GC_CreateThread( ^ /tmp/ecl/src/c/threads/process.d: In function 'mp_exit_process': /tmp/ecl/src/c/threads/process.d:604:1: warning: 'noreturn' function does return Makefile:85: recipe for target 'threads/process.o' failed make[2]: *** [threads/process.o] Error 1 make[2]: Leaving directory '/tmp/ecl/build/c' Makefile:120: recipe for target 'libeclmin.a' failed make[1]: *** [libeclmin.a] Error 2 make[1]: Leaving directory '/tmp/ecl/build' Makefile:70: recipe for target 'all' failed make: *** [all] Error 2
What is the problem here?
(Background: I developed a crosscompiled Windows installer for the (Lisp based) computer algebra system 'Maxima' (using 'Wine' (does ECL work with the emulator 'wine'?)), currently based on CLISP and SBCL - and probably I can support ECL too - but for that I should be able to crosscompile it.
Best regards, Wolfgang
Hello,
Wolfgang Dautermann writes:
Dear ECL-developers,
Can one crosscompile ECL for Windows (on a Linux based system)?
it should, but apparently it doesn't.
Currently it fails - both the latest release and the current GIT - with:
$ ./configure --host=i686-w64-mingw32 $ make [...] i686-w64-mingw32-gcc -DECLDIR=""/u2sr/local/lib/ecl-16.1.2"" -I. -I/tmp/ecl/build -I/tmp/ecl/src/c -I../ecl/gc -DECL_API -DECL_NO_LEGACY -g -O2 -D_THREAD_SAFE -Dmingw32 -c -o threads/process.o threads/process.o.c /tmp/ecl/src/c/threads/process.d:40:22: error: conflicting types for 'GC_CreateThread' extern HANDLE WINAPI GC_CreateThread( ^ In file included from /tmp/ecl/build/ecl/config.h:84:0, from /tmp/ecl/build/ecl/ecl.h:37, from /tmp/ecl/src/c/threads/process.d:20: /tmp/ecl/build/ecl/gc/gc.h:1536:26: note: previous declaration of 'GC_CreateThread' was here GC_API HANDLE WINAPI GC_CreateThread( ^ /tmp/ecl/src/c/threads/process.d: In function 'mp_exit_process': /tmp/ecl/src/c/threads/process.d:604:1: warning: 'noreturn' function does return Makefile:85: recipe for target 'threads/process.o' failed make[2]: *** [threads/process.o] Error 1 make[2]: Leaving directory '/tmp/ecl/build/c' Makefile:120: recipe for target 'libeclmin.a' failed make[1]: *** [libeclmin.a] Error 2 make[1]: Leaving directory '/tmp/ecl/build' Makefile:70: recipe for target 'all' failed make: *** [all] Error 2
What is the problem here?
It seems that we have patched bdwgc (external project) definitions in our project for Windows before the mentioned project did exaclty the same and we have a conflicting declaration.
That is defienetely ECL bug. I've created an issue on GitLab:
https://gitlab.com/embeddable-common-lisp/ecl/issues/288
there is some potential fix (I didn't include it, because of the comment in our own source code - this needs to be verified). If you could check, if the compilation finishes succesfully for you now with the mentioned diff applied, it would be great.
(Background: I developed a crosscompiled Windows installer for the (Lisp based) computer algebra system 'Maxima' (using 'Wine' (does ECL work with the emulator 'wine'?)), currently based on CLISP and SBCL - and probably I can support ECL too - but for that I should be able to crosscompile it.
Best regards, Wolfgang
Best regards, Daniel
On 2016-09-16 10:36, Daniel Kochmański wrote:
That is defienetely ECL bug. I've created an issue on GitLab:
https://gitlab.com/embeddable-common-lisp/ecl/issues/288
there is some potential fix (I didn't include it, because of the comment in our own source code - this needs to be verified). If you could check, if the compilation finishes succesfully for you now with the mentioned diff applied, it would be great.
Thank you very much for the fast response. That seems to solve that issue.
Currently I try to compile with the following shell script:
------------------------------------------------------- LANG=C rm -rf build/ ./configure ABI=32 CFLAGS=-m32 LDFLAGS=-m32 --disable-longdouble --enable-libatomic=included --prefix=$(pwd)/ecl-host make make install export ECL_TO_RUN=$(pwd)/ecl-host/bin/ecl rm -rf build/ ./configure --host=i686-w64-mingw32 --prefix=$(pwd)/ecl-win32 make -------------------------------------------------------
(Adopted from the 'crosscompile for android' section. Do I need to set --with-cross-config to something? There is only a file for android.)
The ecl-host builds well. In the crosscompiled version, it seems that it runs the ecl for the *target* system, which doesnt work at some point (I can provide the complete build log, if necessary, that is just the end, where the error occurs):
[...] ;;; Loading "Z:/home/dauti/ecl/src/cmp/cmpos-features.lsp" File not found.
;;; Unable to execute program "i686-w64-mingw32-gcc";;; Condition;;; Could not spawn subprocess to run ""i686-w64-mingw32-gcc" "--version"".File not found.
;;; Unable to execute program "i686-w64-mingw32-gcc";;; Condition;;; Could not spawn subprocess to run ""i686-w64-mingw32-gcc" "-g" "-O2" "-D_THREAD_SAFE" "-Dmingw32" "-E" "C:/users/dauti/Temp/fa488.c" "-o" "C:/users/dauti/Temp/fa488.i"".;;; System features: NIL ;;; Loading "Z:/home/dauti/ecl/src/cmp/cmpmain.lsp" ;;; Loading "Z:/home/dauti/ecl/src/cmp/proclamations.lsp" ;;; Loading "Z:/home/dauti/ecl/src/cmp/sysfun.lsp"
;;; ;;; Now we are in shape to do something useful. ;;; End of bare.lsp (compile-file "src:lsp;export.lsp" :output-file #P"BUILD:LSP;EXPORT.O.NEWEST" :SYSTEM-P T :C-FILE T :DATA-FILE T :H-FILE T) ;;; ;;; Compiling SRC:LSP;EXPORT.LSP.;;; OPTIMIZE levels: Safety=2, Space=1, Speed=1, Debug=1 ;;; ;;; Compiling (DEFUN EVAL-FEATURE ...). ;;; Compiling (DEFUN DO-READ-FEATURE ...). ;;; Compiling (DEFUN SHARP-+-READER ...). ;;; Compiling (DEFUN SHARP---READER ...). ;;; End of Pass 1.;;; Emitting code for DOLIST. ;;; Emitting code for DOTIMES. ;;; Emitting code for DO/DO*-EXPAND. ;;; Emitting code for EVAL-FEATURE. ;;; Emitting code for DO-READ-FEATURE. ;;; Emitting code for SHARP-+-READER. ;;; Emitting code for SHARP---READER. ;;; Note: ;;; Invoking external command: ;;; i686-w64-mingw32-gcc -I. -I/home/dauti/ecl/build -DECL_API -I/home/dauti/ecl/build/c -g -O2 -D_THREAD_SAFE -Dmingw32 -I/home/dauti/ecl/src/c -c Z:/home/dauti/ecl/build/lsp/export.c -o Z:/home/dauti/ecl/build/lsp/export.o File not found.
Condition of type: SIMPLE-ERROR Could not spawn subprocess to run ""i686-w64-mingw32-gcc" "-I." "-I/home/dauti/ecl/build" "-DECL_API" "-I/home/dauti/ecl/build/c" "-g" "-O2" "-D_THREAD_SAFE" "-Dmingw32" "-I/home/dauti/ecl/src/c" "-c" "Z:/home/dauti/ecl/build/lsp/export.c" "-o" "Z:/home/dauti/ecl/build/lsp/export.o"". Available restarts:
1. (ABORT) ABORT
Top level in: #<process TOP-LEVEL>.
(Notice the Windows/Dos path with drive letters like 'Z:/home/dauti/ecl/build/lsp/export.o'.
The build *does* complete, if I copy ecl_min from the ecl-host build to the build dir: cp ./ecl-host/lib/ecl-16.1.2/ecl_min ./build/ecl_min.exe cp ./ecl-host/lib/ecl-16.1.2/ecl_min ./build/ecl_min after the first 'make' failed and build it (using 'make') a second time. Have I just set a variable wrong (ECL_TO_RUN should be okay? Do I need to set anything else?)
Best regards, Wolfgang
Wolfgang Dautermann writes:
Thank you very much for the fast response. That seems to solve that issue.
This time it took me more time, sorry about that.
Currently I try to compile with the following shell script:
LANG=C rm -rf build/ ./configure ABI=32 CFLAGS=-m32 LDFLAGS=-m32 --disable-longdouble --enable-libatomic=included --prefix=$(pwd)/ecl-host make make install export ECL_TO_RUN=$(pwd)/ecl-host/bin/ecl rm -rf build/ ./configure --host=i686-w64-mingw32 --prefix=$(pwd)/ecl-win32 make
I had to tweak a few flags to start CC on my host, but succeeded.
(Adopted from the 'crosscompile for android' section. Do I need to set --with-cross-config to something? There is only a file for android.)
well, it's pretty target-dependent and involves some guessing and tweaks to the cross-configuration usually. I don't know if defaults work for your target well. Cross compilation isn't well support at this point, but we try to improve that.
The ecl-host builds well. In the crosscompiled version, it seems that it runs the ecl for the *target* system, which doesnt work at some point (I can provide the complete build log, if necessary, that is just the end, where the error occurs):
[...] ;;; Loading "Z:/home/dauti/ecl/src/cmp/cmpos-features.lsp" File not found.
What host are you building on? ("Z:/" looks very strange if its Linux). I doubt that the target binary would run on different operating system at all, it must be something else imho.
(Notice the Windows/Dos path with drive letters like 'Z:/home/dauti/ecl/build/lsp/export.o'.
The build *does* complete, if I copy ecl_min from the ecl-host build to the build dir: cp ./ecl-host/lib/ecl-16.1.2/ecl_min ./build/ecl_min.exe cp ./ecl-host/lib/ecl-16.1.2/ecl_min ./build/ecl_min after the first 'make' failed and build it (using 'make') a second time. Have I just set a variable wrong (ECL_TO_RUN should be okay? Do I need to set anything else?)
I can confirm, that there is problem with ecl_min (I've tried to CC it on my host to windows). It's certainly a bug. Please subscribe to https://gitlab.com/embeddable-common-lisp/ecl/issues/288 (I've reopened the last issue) to be kept posted.
Best regards, Wolfgang
Best regards, Daniel
OK, fixed in the `develop' branch, if you could try the lastest changes and report back if it work for you I would be grateful.
Thanks, Daniel
On 2016-09-20 14:46, Daniel Kochmański wrote:
Cross compilation isn't well support at this point, but we try to improve that.
Hi, (sorry for the late answer)
Thank you. I found some further issues.
What host are you building on? ("Z:/" looks very strange if its Linux). I doubt that the target binary would run on different operating system at all, it must be something else imho.
I am building on a Linux host. But thats was the issue. The compiliation process *did* run the (crosscompiled) binaries using the Linux Kernel feature 'binfmt_misc' (recognize the format and execute it in a user space emulator ("wine"))
When I did a 'chmod 000 /usr/bin/wine', it did produce a 'cross_config' file, which I could save and use with the parameter --with-cross-config (I am attaching the produced file here, if it is correct, perhaps it can be included in the source? I did change ECL_NEWLINE and commmented out ECL_TO_RUN because I want to use my self compiled ecl version.). What does ECL_WORKING_ENVIRON mean?
But this only works, if 'wine' is not available.
ECL seems to test, if a produced executable can be executed - and if this works, it assumes that one is not crosscompiling, right? Suggestion: If the parameter --with-cross-config is used, one wants to crosscompile, that should not be tested then.
Then the build works. 'make install' fails then with:
/usr/bin/install -c c/dpp.exe /home/dauti/ecl/ecl-mingw-root /usr/bin/install: cannot stat 'c/dpp.exe': No such file or directory Makefile:156: recipe for target 'install' failed
That file (dpp.exe) is not produced during the compiliation.
I did install ecl ignoring errors (make -k install) but the compiled FAS-files (build/*.fas) did not seem to get installed too, but that may be because of the errors before.
I then tried to build Maxima with the crosscompiled ecl (using wine, I do that with CLISP and SBCL too). That produced then a internal error - but that may be because the installation was not done correctly or perhaps a issue in 'wine'. Or some values of the cross-config files are wrong?:
;;; Compiling Z:/home/dauti/maxima-code/crosscompile-windows/build/maxima-prefix/src/maxima/src/maxima-package.lisp.;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3, Debug=2 ;;; ;;; End of Pass 1.;;; Internal error: ;;; ** Detected access to an invalid or protected memory address.
Best regards, Wolfgang
On 2016-09-16 10:36, Daniel Kochmański wrote:
Wolfgang Dautermann writes:
Dear ECL-developers,
Can one crosscompile ECL for Windows (on a Linux based system)?
it should, but apparently it doesn't.
Dear ECL-developers,
I think it works now. If someone wants to try it - get http://wolfgang.dautermann.at/ecl/ecl-crosscompiling.tar.gz and follow the instructions from http://wolfgang.dautermann.at/ecl/ecl-crosscompiling/Readme-crosscompiling.t...
This will get & compile libgmp (currently needed for the host-ecl), then ECL (currently the most recent GIT commit from the 'build-clean' branch - and currently fixed to that commit since I need to include a patch (http://wolfgang.dautermann.at/ecl/ecl-crosscompiling/ecl-uname-mingw.patch). Then ECL will be build twice - first for Linux (ecl-host), then for Windows and an installer is created. (tested on Debian/Ubuntu). The crosscompiled ECL was tested with Wine/Linux and Windows10. You can get the compiled installer also from: http://wolfgang.dautermann.at/ecl/ecl-crosscompiling
If you find it useful, feel free to include my code in ECL.
Some minor issues: I build ECL currently with the bytecode compiler (--with-cmp=no) - as the current Windows installer is build. That works, but if I try a compile example from https://common-lisp.net/project/ecl/manual/ch34s06.html it seems to try the native compiler:
(compile-file "hello.lisp")
Condition of type: FILE-ERROR Filesystem error with pathname #P"SYS:CMP.NEWEST".
Shouldnt a bytecode compiler be invoked, if ECL is configured with "--with-cmp=no"?
I can also build it *with* a C compiler (= without "--with-cmp=no" (my idea is to include tdm-gcc (https://sourceforge.net/projects/tdm-gcc/), but the issue is:
During the compiliaton of ECL I need the crosscompiler (i686-w64-mingw32-gcc). It does build fine with that.
But in the installed ECL (when it runs on Windows (or in Wine)) I would need the native compiler (from the tdm-gcc project). ECL seems to use the same compiler (when configured with "./configure --host=i686-w64-mingw32") for building and execution afterwards. Is there a way to specify a distinct compiler (linker, etc.) which will be invoked when the installed ECL runs?
Best regards, Wolfgang
Dear Wolfgang,
I think it works now. If someone wants to try it - get http://wolfgang.dautermann.at/ecl/ecl-crosscompiling.tar.gz and follow the instructions from http://wolfgang.dautermann.at/ecl/ecl-crosscompiling/Readme-crosscompiling.t...
I'll check it out on Monday. Thanks for working on this!
This will get & compile libgmp (currently needed for the host-ecl), then ECL (currently the most recent GIT commit from the 'build-clean' branch
- and currently fixed to that commit since I need to include a patch
(http://wolfgang.dautermann.at/ecl/ecl-crosscompiling/ecl-uname-mingw.patch). Then ECL will be build twice - first for Linux (ecl-host), then for Windows and an installer is created. (tested on Debian/Ubuntu). The crosscompiled ECL was tested with Wine/Linux and Windows10. You can get the compiled installer also from: http://wolfgang.dautermann.at/ecl/ecl-crosscompiling
If you find it useful, feel free to include my code in ECL.
All patches improving portability of ECL are very welcomed.
Some minor issues: I build ECL currently with the bytecode compiler (--with-cmp=no) - as the current Windows installer is build. That works, but if I try a compile example from https://common-lisp.net/project/ecl/manual/ch34s06.html it seems to try the native compiler:
(compile-file "hello.lisp")
Condition of type: FILE-ERROR Filesystem error with pathname #P"SYS:CMP.NEWEST".
Shouldnt a bytecode compiler be invoked, if ECL is configured with "--with-cmp=no"?
Yet it should. If you could make an issue on gitlab it would be great.
I can also build it *with* a C compiler (= without "--with-cmp=no" (my idea is to include tdm-gcc (https://sourceforge.net/projects/tdm-gcc/), but the issue is:
During the compiliaton of ECL I need the crosscompiler (i686-w64-mingw32-gcc). It does build fine with that.
But in the installed ECL (when it runs on Windows (or in Wine)) I would need the native compiler (from the tdm-gcc project). ECL seems to use the same compiler (when configured with "./configure --host=i686-w64-mingw32") for building and execution afterwards. Is there a way to specify a distinct compiler (linker, etc.) which will be invoked when the installed ECL runs?
I'm working on this in a separate branch, but it won't be included in upcoming release.
In general I want to create a convenient way for registering different compilers (mainly for cross-compilation, but this will work for your usecase as well).
Best regards, Wolfgang
Best regards, Daniel
Your patch added to the develop branch.
Best regards, Daniel
On 2016-11-12 18:04, Daniel Kochmański wrote:
Shouldnt a bytecode compiler be invoked, if ECL is configured with "--with-cmp=no"?
Yet it should. If you could make an issue on gitlab it would be great.
Hm. If both compilers (bytecode & C) are enabled, how can one select which one should be used? Or which should be the default?
Somewhere I found that "(ext::install-bytecodes-compiler)" works. This should probably be added to the manual where compiliation mentioned (https://common-lisp.net/project/ecl/manual/ch34s06.html)
That seems to work now. And if I call ecl with ecl.exe --eval "(ext::install-bytecodes-compiler)" [further parameters] the bytecode compiler is called by default. Perhaps I am able to crosscompile Maxima with it. There is no easier command line option (or configuration value somewhere) for that?
I'm working on this in a separate branch, but it won't be included in upcoming release.
In general I want to create a convenient way for registering different compilers (mainly for cross-compilation, but this will work for your usecase as well).
Thank you.
Best regards, Wolfgang
On 2016-11-14 16:20, Daniel Kochmański wrote:
Your patch added to the develop branch.
Best regards, Daniel
Thank you. That branch I could not (cross)compile. The native compiliation works, but when I crosscompile, I get error messages, where I was not able to find the reason:
First the 'preprocessor' emits some warnings:
dpp: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/symbol.d -> c/symbol.c dpp: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d -> c/package.c
Unknown symbol: si::*ignore-package-locks*
Unknown symbol: si::*ignore-package-locks*
And later there are compilation errors because of this unknown symbol:
Compiling c/package.o... In file included from ./ecl/ecl.h:89:0, from /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d:16: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d: In function ‘ecl_rename_package’: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d:250:22: error: ‘unknown’ undeclared (first use in this function) @'si::*ignore-package-locks*') == ECL_NIL) { ^ ./ecl/stacks.h:59:47: note: in definition of macro ‘ECL_SYM_VAL’ # define ECL_SYM_VAL(env,s) (ecl_bds_read(env,s))
(further errors follow). ^
But the "build-clean" branch (which you recommended, that I should use) is okay (with that patch) - no errors occur there.
Best regards, Wolfgang
Wolfgang Dautermann writes:
On 2016-11-14 16:20, Daniel Kochmański wrote:
Your patch added to the develop branch.
Best regards, Daniel
Thank you. That branch I could not (cross)compile. The native compiliation works, but when I crosscompile, I get error messages, where I was not able to find the reason:
First the 'preprocessor' emits some warnings:
dpp: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/symbol.d -> c/symbol.c dpp: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d -> c/package.c
Unknown symbol: si::*ignore-package-locks*
Unknown symbol: si::*ignore-package-locks*
And later there are compilation errors because of this unknown symbol:
Compiling c/package.o... In file included from ./ecl/ecl.h:89:0, from /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d:16: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d: In function ‘ecl_rename_package’: /home/dauti/Software/ecl-crosscompiling/build/ecl-prefix/src/ecl/src/c/package.d:250:22: error: ‘unknown’ undeclared (first use in this function) @'si::*ignore-package-locks*') == ECL_NIL) { ^ ./ecl/stacks.h:59:47: note: in definition of macro ‘ECL_SYM_VAL’ # define ECL_SYM_VAL(env,s) (ecl_bds_read(env,s))
(further errors follow). ^
It seems, that your host ECL has different version than sources you compile. Make sure, that you remove build/ directory (for instance with make distclean), rebuild host ECL and cross-compile target ECL with the new host.
But the "build-clean" branch (which you recommended, that I should use) is okay (with that patch) - no errors occur there.
develop branch has now build-clean branch merged. Please use it instead now. Thanks.
Hm. If both compilers (bytecode & C) are enabled, how can one select which one should be used? Or which should be the default?
as you have mentioned, you should use ext:install-bytecodes-compiler or ext:install-c-compiler
Somewhere I found that "(ext::install-bytecodes-compiler)" works. This should probably be added to the manual where compiliation mentioned (https://common-lisp.net/project/ecl/manual/ch34s06.html)
That seems to work now. And if I call ecl with ecl.exe --eval "(ext::install-bytecodes-compiler)" [further parameters] the bytecode compiler is called by default. Perhaps I am able to crosscompile Maxima with it. There is no easier command line option (or configuration value somewhere) for that?
Not yet. bytecodes compiler is usually installed by default on Windows though.
Best regards, Daniel
On 2016-11-16 16:57, Daniel Kochmański wrote:
It seems, that your host ECL has different version than sources you compile. Make sure, that you remove build/ directory (for instance with make distclean), rebuild host ECL and cross-compile target ECL with the new host.
Thanks. It works now without problems (with the 'develop' branch). I updated the build system and documentation on http://wolfgang.dautermann.at/ecl/
Best regards and thank you for the support, Wolfgang
-
Daniel Kochmański -
Wolfgang Dautermann