On Tue, Sep 8, 2009 at 2:19 AM, John Pallisterjohn@synchromesh.com wrote:
Hello list,
I'm interested in trying to get ABCL running (and preferably REPL-ing & compiling) on a certain cloud computing platform that has a JVM with no access to a file system.
The Wiki page for JSR-223 and the sample code both mention that "compilation using the runtime compiler has been removed due to inconsistencies with evaluation and file-based compilation", since May '09. Browsing the mailing list archives suggests that people have been working on removing the need for temporary files since then, but I can't find any mention of "inconsistencies". The thread "Save-image w/Serialization - progress report" from April ends with Sig. Stalla in the process of implementing a byte-array-output-stream, then silence...
Hello John,
Sig. Stalla sounds pompous, you can call me Alessio ;) Those you cite are really 2 separate issues.
1. For the JSR-223 part, it used to use the cl function COMPILE to compile code using the runtime compiler. However its behavior differs from COMPILE-FILE in ways that could result in unexpected behavior, so now the ABCL script engine uses COMPILE-FILE on its own, by saving the code to compile to temporary files. This could be fixed by using a primitive that works like COMPILE-FILE but is able to read from any stream, included from a string in memory. I don't exclude such a thing already exists, and only needs to be polished and made public.
2. However in general, JSR-223 or not, the ABCL compiler currently always uses files at least when compiling local functions, even if you use the runtime compiler. That is what you see in the serialization thread. Changing this is more problematic, since it probably requires changing the way compiled Lisp code is loaded and the instructions emitted by the compiler to load compiled local functions, and I'm afraid also the order in which the compiler does some things (wild guess here!). I'm also interested in this change, but I don't have much time to work on it.
Can someone comment on the state of runtime compilation, preferably entirely in-memory without reference to a file system?
Basically I'd like to be able to compile stuff to a byte array, store the array as a blob in a memcache and/or other datastore, then pull it out again as required and load it via a custom classloader. Is this doable, or would it be easier to port ParenScript to Clojure (which is currently my Plan B)?
A relatively quick fix would be to implement a virtual file API that writes either on disk or in memory (in a dummy, virtual filesystem). The compiler would be only changed slightly to use the more high-level API instead of direct file access. Primitives to load compiled code would need to look at some variable to determine where to load code from. The virtual filesystem could be exposed to the user so it could be persisted & restored across sessions.
I think this approach is not enough to solve the problems with serialization (because the virtual fs needs to be manually saved/restored, since loaded code does not know where it is loaded from), but it's probably ok for situations like yours. Note that you'd have to use COMPILE-FILE and not COMPILE with the virtual fs, in order to have a known entry point (the virtual file name) when loading code later. Also maybe a finer-grained control on the virtual fs could be given to the user, so you'd be able to decide the life span of the fs.
If others have ideas in this regard, please share. I'd like to avoid Plan B (having users resort to the competition :D)
Bye, Alessio
Cheers, and it's great to see the ABCL project still going strong!
John :^P
P.S. If anyone is going to be in Hamburg at the weekend and wants to chat about this stuff, that'd be great! -- John Pallister john@johnp.net john@synchromesh.com
armedbear-devel mailing list armedbear-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel