[armedbear-devel] asdf and windows vs unix on abcl
ABCL is special in that it can run on either Windows or Unix. The problem is that ASDF 2.017 determines which of Windows or Unix at compile-time, which causes confusion when ABCL's builtin ASDF is compiled on Windows and used on Unix or vice-versa. And I can tell your binary was definitely produced on Windows. (I found this issue while testing cl-launch. I also noticed that you fixed your ext:*command-line-argument-list* albeit in a necessarily incompatible way.) I suppose the solution is that the test of unixness vs windowsness should happen at runtime. Crazy. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
On 20 October 2011 19:00, Faré <fahree@gmail.com> wrote:
ABCL is special in that it can run on either Windows or Unix.
..with the same binary, even, that is.
I suppose the solution is that the test of unixness vs windowsness should happen at runtime. Crazy.
I don't see anything crazy about such tests being done at runtime rather than at compile-time.
On Oct 20, 2011, at 18:00 , Faré wrote:
ABCL is special in that it can run on either Windows or Unix.
The problem is that ASDF 2.017 determines which of Windows or Unix at compile-time, which causes confusion when ABCL's builtin ASDF is compiled on Windows and used on Unix or vice-versa. And I can tell your binary was definitely produced on Windows.
(I found this issue while testing cl-launch. I also noticed that you fixed your ext:*command-line-argument-list* albeit in a necessarily incompatible way.)
I suppose the solution is that the test of unixness vs windowsness should happen at runtime. Crazy.
We place :WINDOWS and :UNIX in *FEATURES* depending on where we end up executing, so forms like (when (find :unix *features*) (run-process "ls")) should do the right sorts of things. If you have other suggestions about how to manage this, please make them. Mark <evenson@panix.com>
I suppose the solution is that the test of unixness vs windowsness should happen at runtime. Crazy.
We place :WINDOWS and :UNIX in *FEATURES* depending on where we end up executing, so forms like
(when (find :unix *features*) (run-process "ls"))
should do the right sorts of things. If you have other suggestions about how to manage this, please make them.
You're doing the right thing. I suppose you're also doing some magic to properly parse C:\foo\bar\ on Windows and /c/foo/bar on Unix. ASDF just wasn't prepared to that (and neither was XCVB). Please test 2.017.22, I'd like to release it as 2.018, at the end of the month. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org No man would listen to you talk if he didn't know it was his turn next. — Edgar Watson Howe
On Oct 21, 2011, at 03:37 , Faré wrote:
I suppose the solution is that the test of unixness vs windowsness should happen at runtime. Crazy.
We place :WINDOWS and :UNIX in *FEATURES* depending on where we end up executing, so forms like
(when (find :unix *features*) (run-process "ls"))
should do the right sorts of things. If you have other suggestions about how to manage this, please make them.
You're doing the right thing. I suppose you're also doing some magic to properly parse C:\foo\bar\ on Windows and /c/foo/bar on Unix.
The underlying JVM abstraction (java.io.File) helps a bit with smoothing over these differences, but the essential difference is that the DOS drive letter (e.g. "C:") is mapped to the Pathanme DEVICE component IFF we're executing on Windows. And the ABCL Pathanme name string *always* canonicalizes the directory separator to forward slashes ("/") regardless of runtime platform. In general, we try to DWIM on most operations but there are some corner cases, such as deserializing name strings on different platforms where the implementation can run into lacunae.
ASDF just wasn't prepared to that (and neither was XCVB). Please test 2.017.22, I'd like to release it as 2.018, at the end of the month.
Hmmm. We were gonna release abcl-1.0.0 on October 22, 2011 for ECLM. Should we trying upgrading to 2.017.22 before the release? I need to look at the changes in ASDF…
ASDF just wasn't prepared to that (and neither was XCVB). Please test 2.017.22, I'd like to release it as 2.018, at the end of the month.
Hmmm. We were gonna release abcl-1.0.0 on October 22, 2011 for ECLM. Should we trying upgrading to 2.017.22 before the release? I need to look at the changes in ASDF…
Oh wow, that sounds great. Yes, please upgrade -- but test before you release. ASDF 2.017.22 passes all my tests, and compiles XCVB and its many dependencies, but that doesn't mean you won't run into problems, does it? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Dear ABCL hackers, congratulations for releasing 1.0!!! Doing a last round of cleanups before release of ASDF, I find that I must test features at runtime for finding the implementation-identifier, too. The next question of course is: should it matter that we're running on linux-x64 or not? It might, if people ever use CFFI to link to actual C code via JNI or some such. Does ABCL have this capability? If yes (or you intend to eventually/soon), then I'll leave OS and machine type in the identifier. If not, then I'll both expand to Java (or does the specific Java-1.x tag matter?). —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
On Oct 24, 2011, at 18:16 , Faré wrote:
Dear ABCL hackers,
congratulations for releasing 1.0!!!
Doing a last round of cleanups before release of ASDF, I find that I must test features at runtime for finding the implementation-identifier, too.
The next question of course is: should it matter that we're running on linux-x64 or not? It might, if people ever use CFFI to link to actual C code via JNI or some such. Does ABCL have this capability? If yes (or you intend to eventually/soon), then I'll leave OS and machine type in the identifier. If not, then I'll both expand to Java (or does the specific Java-1.x tag matter?).
The referenced [patch will enable CFFI running under ABCL to load jna.jar from Maven repositories over the internetwork][1]. [1]: http://trac.common-lisp.net/armedbear/ticket/176 =---- For most purposes, what the operating environment is executing the hosted JVM should be irrelevant for users of the implementation. jna.jar packages native shared objects in a variety of formats, and has reasonable code for finding the correct implementation to dynamically link, so for ABCL just defer to the use of ABCL-ASDF:RESOLVE-DEPENDENCIES. --- There seems to be something a little funky with the ABCL JAR-PATHNAME under ASDF-2.017.22, so we expect to patch for abcl-1.0.1 (within a month), so we're right on track with you. How can I test XCVB? --- To find what platform the hosted JVM is executing under, I might suggest some variant of CL-USER> (when (find :unix *features* ) (run-program "uname" '("-a")))
On Oct 24, 2011, at 18:55 , Mark Evenson wrote:
[…]
To find what platform the hosted JVM is executing under, I might suggest some variant of
CL-USER> (when (find :unix *features* ) (run-program "uname" '("-a")))
[…] Well, we seem to have symbol for the underlying processor in our *FEATRURES* iist: ; SLIME 2011-09-27 CL-USER> *features* (:SWANK :QUICKLISP :ASDF2 :ASDF :X86-64 :JAVA-1.6 :ARMEDBEAR :ABCL :COMMON-LISP :ANSI-CL :UNIX :SUNOS :SOLARIS :CDR6) CL-USER> You can depend on this remaining present.
For most purposes, what the operating environment is executing the hosted JVM should be irrelevant for users of the implementation. jna.jar packages native shared objects in a variety of formats, and has reasonable code for finding the correct implementation to dynamically link, so for ABCL just defer to the use of ABCL-ASDF:RESOLVE-DEPENDENCIES.
So the jar contains multiple .so, one for each supported platform, and dispatches at runtime? e.g. a windows .dll, a mac .library, linux .so for x64 x86 (and why not arm hppa sparc, etc.), plus maybe a .so for solaris or bsd x64, sparc64, etc. Must be both cool for users a somewhat "interesting" for builders. In this case indeed java as the OS and architecture would fit.
There seems to be something a little funky with the ABCL JAR-PATHNAME under ASDF-2.017.22, so we expect to patch for abcl-1.0.1 (within a month), so we're right on track with you. How can I test XCVB?
Don't forget to send me the patch! XCVB support for ABCL is quite minimal: * XCVB supports ABCL as a target (though not currently as a host; portability to SBCL, CCL and CLISP is already horrible). * XCVB does not support building an executable with ABCL yet. * due to high ABCL start time, the normal XCVB building of each file in a new process is painfully slow. * XCVB can still output an ASDF for ABCL to use, but that's not very useful. In the near future, I'll add support for "reverse slave" whereby the master who calls XCVB gets back instructions from XCVB to do the building itself; no need for a clumsy roundtrip through ASDF anymore. So XCVB can become a stand-in replacement for ASDF even when it doesn't bring any particular advantage. To test XCVB, grab the latest release tarball with all the dependencies, compile with SBCL (preferred) with make install, and try building, say, to compile fare-utils, to make it simple, and watch it slog through: xcvb simple-build --build fare-utils \ --lisp-implementation abcl --verbosity 99 I just tried, and I'm pleasantly surprised it just worked (though very slowly), though I haven't touched ABCL support in a while. Can ABCL somehow start a fresh image for cheap in the same Java heap, or even better, fork a CL image within the same Java heap? That would make ABCL much more useful as an XCVB backend.
To find what platform the hosted JVM is executing under, I might suggest some variant of
CL-USER> (when (find :unix *features* ) (run-program "uname" '("-a")))
Is that useful? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I love deadlines. I love the whooshing sound they make as they fly by. — Douglas Adams
On Oct 24, 2011, at 19:21 , Faré wrote: […]
Can ABCL somehow start a fresh image for cheap in the same Java heap, or even better, fork a CL image within the same Java heap? That would make ABCL much more useful as an XCVB backend.
No, the usual JVM implementations don't have this sort of capability. If our abstractions of specials, streams, etc. are managed correctly, we should be able to run threads for you quite cheaply. I'm working on a wrapping of the JVM Executor service. Stay tuned.
To find what platform the hosted JVM is executing under, I might suggest some variant of
CL-USER> (when (find :unix *features* ) (run-program "uname" '("-a")))
Is that useful?
Not sure. You have to interpolate the string output, but in the grand scheme of things, that isn't hard, is it? What was the question? ;) Mark <evenson@panix.com> -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On Mon, Oct 24, 2011 at 7:39 PM, Mark Evenson <evenson.not.org@gmail.com> wrote:
On Oct 24, 2011, at 19:21 , Faré wrote: […]
Can ABCL somehow start a fresh image for cheap in the same Java heap, or even better, fork a CL image within the same Java heap? That would make ABCL much more useful as an XCVB backend.
No, the usual JVM implementations don't have this sort of capability. If our abstractions of specials, streams, etc. are managed correctly, we should be able to run threads for you quite cheaply. I'm working on a wrapping of the JVM Executor service. Stay tuned.
I believe Faré's reason for starting a new image is to have it clean of any side-effects of previous compilations. Strictly speaking, this would only be possible if the JVM supported fork() or save-image, but unfortunately it doesn't and will probably never do. However, I have an idea for a compromise that would not get you a truly fresh new image, but could nevertheless create a new environment clean enough for most use cases. I don't have the time to go into details now, but that's definitely something I've been wanting to experiment with for some time. Alessio
On Oct 24, 2011, at 22:42 , Alessio Stalla wrote […]
I believe Faré's reason for starting a new image is to have it clean of any side-effects of previous compilations. Strictly speaking, this would only be possible if the JVM supported fork() or save-image, but unfortunately it doesn't and will probably never do. However, I have an idea for a compromise that would not get you a truly fresh new image, but could nevertheless create a new environment clean enough for most use cases. I don't have the time to go into details now, but that's definitely something I've been wanting to experiment with for some time.
Certainly sounds interesting. A FRESH-IMAGE contrib perhaps?
On Mon, Oct 24, 2011 at 10:51 PM, Mark Evenson <evenson.not.org@gmail.com> wrote:
On Oct 24, 2011, at 22:42 , Alessio Stalla wrote
[…]
I believe Faré's reason for starting a new image is to have it clean of any side-effects of previous compilations. Strictly speaking, this would only be possible if the JVM supported fork() or save-image, but unfortunately it doesn't and will probably never do. However, I have an idea for a compromise that would not get you a truly fresh new image, but could nevertheless create a new environment clean enough for most use cases. I don't have the time to go into details now, but that's definitely something I've been wanting to experiment with for some time.
Certainly sounds interesting. A FRESH-IMAGE contrib perhaps?
I will think about it. I should also note that with appropriate classloader manipulation you can already start multiple ABCL images in the same JVM, but that doesn't help much because each one will load from scratch and take a long time.
participants (5)
-
Alessio Stalla -
Faré -
Mark Evenson -
Mark Evenson -
Ville Voutilainen