As pointed out, Alessio added some primitives required to add in-memory compilation.
However, the rest of the thread seems to mix a number of issues and their progress. As a result, I thought a summary might be in order.
There were several things "in-memory" versus "using files":
1. Compilation of functions in our JSR-223 support (Alessio changed ABCL's behaviour here) 2. Compilation of lisp forms into compiled functions 3. Compilation of lisp files into fasls
With respect to (1): This is where Alessio changed ABCL's behaviour to use COMPILE instead of COMPILE-FILE, eliminating the use of a single temporary file.
With respect to (2): Currently, ABCL uses a temporary file for each "compiland" (FLET/LABELS defined function, LAMBDA, DEFUN) in the forms to be compiled. This file is created with a ".cls" extention. After it's written, it gets loaded into the running jvm by loading the bytes in a byte array. That byte array is then resolved to a class. ABCL uses its own class loader to do so. Because the the bytes get written and read without this being required by the Java class loader, there should not be a need to write the bytes out to a file. After the file has been read into memory, the temporary file gets deleted.
With respect to (3): Each "compiland" in a file to be compiled leads to a ".cls" file, which is created as a temp file, then read into memory and after that written into the ".abcl" file - which as noted is in fact a .zip file. The intermediate temp file is then deleted. This process too takes much more I/O (what if you're compiling to a network drive with this kind of I/O?!) then required.
It's with respect to items (2) and (3) that I'd like to see things changed. In case of (2) there's no reason to send output to any type of disc, while in case (3) it would be beneficial to write our output straight into the ZIP file, instead of using intermediary files.
I'm looking for a good solution to both cases. One of the things I came up with is some kind of "storage factory" which would be a function to be called which either generates a stream sending output to the ZIP, or an in-memory stream or a file-backed stream if that's still required.
So, what are your comments to that?
Bye,
Erik.
Hi Erik,
Thanks for that great summary. It does indeed answer all my questions about in-memory compilation and temporary files.
It does indeed sound like some sort of "map of named streams" abstraction would be better than relying on the presence of a file system, and that indeed the use of temporary files can/should be done away with altogether.
I am very busy with my "day job" (aren't we all?) but I hope to find time soon to investigate how ABCL could support compilation on Google App Engine. To this end, I could try and find a "good solution" that meets everyone's needs.
And I'd like to say again, it's great to see ABCL thriving and improving.
Cheers,
John :^P
On Wed, 9 Sep 2009 at 22:39:44 +0200, Erik Huelsmann wrote:
I'm looking for a good solution to both cases. One of the things I came up with is some kind of "storage factory" which would be a function to be called which either generates a stream sending output to the ZIP, or an in-memory stream or a file-backed stream if that's still required.
So, what are your comments to that?
The original reason, way back in the first days of ABCL's compiler, for writing the output of COMPILE to a file, was to facilitate debugging the compiler.
COMPILE basically created the output class file, then fed it to LOAD- COMPILED-FILE, which, particularly in the early days, might fail to load the class file because of a class verification error.
When this happened, it was then easy enough to find the offending class file on disk and attack it with tools that knew about class files to figure out what was wrong with it. (If I remember correctly, in the *very* early days the temporary file always had the same file name, to make things even easier.)
This sort of failure probably doesn't happen much any more, but it might be handy to keep the old behavior around in some form as an option, just in case.
-Peter
armedbear-devel@common-lisp.net
-
Erik Huelsmann -
John Pallister -
Peter Graves