The spec allows functions in a fasl to hard code references to each other, except when declared not-inline. Currently, we take advantage of that fact only a little bit: the only bit we eliminate is the function lookup from the symbol. Anything else works just like function calls which are not in the same file. The mail below comes from my long-standing desire to take better advantage of the room offered by the spec.
What things would I like to improve on?
1. We only inline backward referenced functions in the same fasl 2. We parse the argument list of each function call with respect to keywords and optional parameters 3. We store the names of symbols referenced in many .cls files each of which gets separately zipped 4. We store (cache) function references to inlined functions as objects
What's the basic idea?
The idea is to shift paradigms: instead of compiling each Lisp function into a class, we will start compiling each lisp function to a Java function. A FASLs would become one or more classes -- depending on if everything fits into a single class or not. The idea is by the way not to change the way we model functions in symbol function slots, but instead to change the way everything is stored in a fasl.
Does it address the items mentioned above?
1. Since class files are written once the class is complete, we have to delay serializing the lisp functions to disk until they're all known. This means that we will know all functions defined in a fasl before it's serialized. With an appropriate linker phase, we could easily resolve all forward referenced function calls.
2. Because lisp functions have become Java functions, we can't directly expose them to the Lisp world anymore. This basically creates the option to have an 'internal function signature' and an 'external function signature'. SBCL has this: XEPs -- eXternal Entry Points. These entry points into a function sort the arguments into the right order before calling the internal entry point. Code which is compiled into the same fasl pre-sorts the arguments at compile time (when possible) and calls the internal entry point -- eliminating the need to sort keyword parameters.
3. By having a single (or fewer) class files, we can achieve higher re-use ratios of the same constants which would otherwise be included in many-many .cls files.
4. Because function calls to 'inline' functions will become calls to sibling methods, they will become clear candidates for inlining to the JIT. With the object references, it is not clear this is an option.
What I think needs to happen to get this designed:
a. The file compiler and function compiler should be based on a shared 'method compiler' which takes its context from the caller instead of the existing way of basing the file compiler on the function compiler.
b. We need to implement a linker phase separate from the pass2 phase, which can re-order arguments and call sibling methods instead of Lisp Function objects.
c. We need to find a way to correctly handle the interaction between the successive IN-PACKAGE, DEFPACKAGE, EVAL-WHEN, etc, forms appearing in the input file and the initialization of fields in the resulting class file.
d. We need a way to expose the external entry points to the lisp world.
e. We need to find a way to split fasls over multiple class files.
Comments?
Bye,
Erik.