Dear list,
from within a REPL, oftentimes, you can save the current status and create a file that’ll allow you to continue where you are now at a later point in time. This file could be a standalone executable or a core file that requires a lisp environment to run (e.g. via `lisp -core …`). These two cases can be subdivided into four overall use cases.
(exe-repl) When the executable is run, you want it to print a banner and take you to a REPL. Maybe you’ve created this file after applying a patch to your lisp and you don’t want to do that over and over again. (exe-application) You’ve created an executable that should not print a banner and run an entirely new main() function.
(core-repl) Analogous to the exe-repl case, but smaller at the cost of requiring a lisp to run (core-application) Analogous to the exe-application case (not sure if anyone does this).
The function `uiop:dump-image` can be used to create images, where ‘image’ is an umbrella term for both standalone executables and cores. With :executable t, it creates the former; with :executable nil, the latter. The image is only told whether we want a REPL or application type image once it’s run and uiop:restore-image hands control over to what uiop:*image-entry-point*.
That seems to work for some lisps, but for CMU CL it does not. CMU CL’s save-image provides options like
:process-command-line t/nil :print-herald t/nil
Whether the herald/banner is printed on startup is decided when save-image is called. It cannot be changed later. If you’re writing an application with a main method, and you don’t pass :print-herald nil, the banner will be printed before your application does anything else. If you want to read a list of options and arguments from the command line in your main method, this will not work if you passed :process-command-line nil to save-image, since then the command line arguments will be frozen (i.e., equal what they were when you called save-image).
This suggests that for uiop:dump-image to cover relevant use cases on CMU CL, it will have to learn about the repl/application distinction, e.g. by growing another option.
I’ve created a sample application that illustrates this and put it here(*):
https://github.com/pipping/cl-echo
echo.lisp does what its name suggests: It tells you what it is that you passed to it. If I want to create an exe-application type image from it, on sbcl e.g. that’ll work with uiop:dump-image and I can even use asdf’s program-op (please see build.lisp for how)
For CMU CL, (uiop:dump-image :executable t) and asdf's program-op do something that I do not want: They leave the command line arguments frozen so that the resulting executable does not return what I want it to. I need something that’s different from what asdf’s program-op in build.lisp does (in particular, not pass :process-command-line nil), as is shown in build-cmucl.lisp.
Were I to pass :print-herald nil with :executable t in uiop:dump-image on CMU CL, I’d make the exe-application case work as expected but not the exe-repl case. Hence my claim that we need a new option.
If we add another option to dump-image, the question is how asdf's image-op and program-op should behave (here, the word image refers to core files only, as far as I can tell). Will we need new options here, too? If so, what should the defaults be?
This write-up is the result of a conversation with Robert. Thanks again, Robert!
I’d welcome feedback.
Elias
(*) The current version at the time of this writing is
https://github.com/pipping/cl-echo/tree/faea91b6aa14ea81f2130edef3f6abccbd5f...
so if my notes don’t seem to make any sense, please give that revision a try.
ASDF distinguishes image-op from program-op. The former is supposed to create an image from which you can resume development. The latter is supposed to create a self-contained executable application that can hopefully be delivered with minimal additional support files (depending on the implementation). Both invoke dump-image (or create-image, on ECL & co), but with slightly different options.
I admit that CMUCL has fallen completely out of my cache, because it requires a 32-bit compiler for dumping executables, and recent Linux distributions don't bother with that anymore. I remember it had command-line issues, but I thought they were resolved, and you could *somehow* access unprocessed arguments if you used :process-command-line nil. If not, I'd say that's a bug to be resolved with the CMUCL maintainer, if still active.
program-op should definitely disable print-herald, and it should definitely disable CMUCL's processing of -eval flags.
image-op should probably preserve the herald and processing of -eval flags. It need not use :executable t though that might confuse poor cl-launch.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Men stumble over the truth from time to time, but most pick themselves up and hurry off as if nothing happened. — Winston Churchill
On Fri, Sep 16, 2016 at 7:19 AM, Elias Pipping pipping.elias@icloud.com wrote:
Dear list,
from within a REPL, oftentimes, you can save the current status and create a file that’ll allow you to continue where you are now at a later point in time. This file could be a standalone executable or a core file that requires a lisp environment to run (e.g. via `lisp -core …`). These two cases can be subdivided into four overall use cases.
(exe-repl) When the executable is run, you want it to print a banner and take you to a REPL. Maybe you’ve created this file after applying a patch to your lisp and you don’t want to do that over and over again. (exe-application) You’ve created an executable that should not print a banner and run an entirely new main() function.
(core-repl) Analogous to the exe-repl case, but smaller at the cost of requiring a lisp to run (core-application) Analogous to the exe-application case (not sure if anyone does this).
The function `uiop:dump-image` can be used to create images, where ‘image’ is an umbrella term for both standalone executables and cores. With :executable t, it creates the former; with :executable nil, the latter. The image is only told whether we want a REPL or application type image once it’s run and uiop:restore-image hands control over to what uiop:*image-entry-point*.
That seems to work for some lisps, but for CMU CL it does not. CMU CL’s save-image provides options like
:process-command-line t/nil :print-herald t/nil
Whether the herald/banner is printed on startup is decided when save-image is called. It cannot be changed later. If you’re writing an application with a main method, and you don’t pass :print-herald nil, the banner will be printed before your application does anything else. If you want to read a list of options and arguments from the command line in your main method, this will not work if you passed :process-command-line nil to save-image, since then the command line arguments will be frozen (i.e., equal what they were when you called save-image).
This suggests that for uiop:dump-image to cover relevant use cases on CMU CL, it will have to learn about the repl/application distinction, e.g. by growing another option.
I’ve created a sample application that illustrates this and put it here(*):
https://github.com/pipping/cl-echo
echo.lisp does what its name suggests: It tells you what it is that you passed to it. If I want to create an exe-application type image from it, on sbcl e.g. that’ll work with uiop:dump-image and I can even use asdf’s program-op (please see build.lisp for how)
For CMU CL, (uiop:dump-image :executable t) and asdf's program-op do something that I do not want: They leave the command line arguments frozen so that the resulting executable does not return what I want it to. I need something that’s different from what asdf’s program-op in build.lisp does (in particular, not pass :process-command-line nil), as is shown in build-cmucl.lisp.
Were I to pass :print-herald nil with :executable t in uiop:dump-image on CMU CL, I’d make the exe-application case work as expected but not the exe-repl case. Hence my claim that we need a new option.
If we add another option to dump-image, the question is how asdf's image-op and program-op should behave (here, the word image refers to core files only, as far as I can tell). Will we need new options here, too? If so, what should the defaults be?
This write-up is the result of a conversation with Robert. Thanks again, Robert!
I’d welcome feedback.
Elias
(*) The current version at the time of this writing is
https://github.com/pipping/cl-echo/tree/faea91b6aa14ea81f2130edef3f6abccbd5f...
so if my notes don’t seem to make any sense, please give that revision a try.
On 16 Sep 2016, at 14:55, Faré fahree@gmail.com wrote:
ASDF distinguishes image-op from program-op. The former is supposed to create an image from which you can resume development. The latter is supposed to create a self-contained executable application that can hopefully be delivered with minimal additional support files (depending on the implementation). Both invoke dump-image (or create-image, on ECL & co), but with slightly different options.
I admit that CMUCL has fallen completely out of my cache, because it requires a 32-bit compiler for dumping executables, and recent Linux distributions don't bother with that anymore. I remember it had command-line issues, but I thought they were resolved, and you could *somehow* access unprocessed arguments if you used :process-command-line nil. If not, I'd say that's a bug to be resolved with the CMUCL maintainer, if still active.
I’ve talked to Raymond Toy about this approximately a week ago here:
https://gitlab.common-lisp.net/cmucl/cmucl/issues/29
and he convinced me that what CMU CL does is not a bug.
Elias
-
Elias Pipping -
Faré