This weekend, I committed an optimization which shows 4 - 12% performance gain on the CL-BENCH common lisp benchmarking toolset tests for CLOS. However, the test collects performance results from a high number of invocations. Low numbers of invocations should show even better improvements.
Profiling one of the performance tests for CL-BENCH which shows huge difference with CLISP, it turned out that our CHECK-INITARGS function in the MOP package is taking 40% of the object instantiation time consumed by MAKE-INSTANCE. After the first invocation, it's possible to replace this function by a lookup in a cache.
The same applies to args checking for some of the other functions defined to object instantiation and class modification. However, the time required to build this into our CLOS in a structured manner, exceeds "just" a few evenings, meaning that I probably will be delaying this task until I have a more time to undertake this effort.
Ofcourse, I remain on the look out for possibilities to increase the performance of our clos implementation. An important input are the issues you experience with our performance, especially if you have example code to demonstrate it.
Bye,
Erik.
Steve,
On Sun, Oct 11, 2009 at 11:10 PM, Erik Huelsmann ehuels@gmail.com wrote:
This weekend, I committed an optimization which shows 4 - 12% performance gain on the CL-BENCH common lisp benchmarking toolset tests for CLOS. However, the test collects performance results from a high number of invocations. Low numbers of invocations should show even better improvements.
With the bit above, I meant to ask you to run your test suite again and report back with the newest timings. I hope we have brought the timings from the first and second runs closer: by reducing the first run.
Bye,
Erik.
Erik Huelsmann wrote:
Steve,
On Sun, Oct 11, 2009 at 11:10 PM, Erik Huelsmann ehuels@gmail.com wrote:
This weekend, I committed an optimization which shows 4 - 12% performance gain on the CL-BENCH common lisp benchmarking toolset tests for CLOS. However, the test collects performance results from a high number of invocations. Low numbers of invocations should show even better improvements.
With the bit above, I meant to ask you to run your test suite again and report back with the newest timings. I hope we have brought the timings from the first and second runs closer: by reducing the first run.
Bye,
Erik.
They are closer. First run, 16 seconds. Second run, 13 seconds.
2009/10/12 Erik Huelsmann ehuels@gmail.com:
Profiling one of the performance tests for CL-BENCH which shows huge difference with CLISP, it turned out that our CHECK-INITARGS function in the MOP package is taking 40% of the object instantiation time consumed by MAKE-INSTANCE. After the first invocation, it's possible to replace this function by a lookup in a cache.
I'm not all that surprised that check-initargs shows up with such a high percentage. That's what Peter Graves warned about, being the reason why the checking has been disabled thus far. I've been aware of the need for a cache of some kind, but I'd prefer getting the generic keyword argument checking to a working state before doing optimizations (well, we can call them performance improvements if that sounds nicer :)). We check them pretty well for the methods invoked by make-instance, but that's about it. We also have some other method invocation failures in the ansi-tests.
I am interested in hearing ideas as to how the cache would be keyed and what exactly would it contain. I haven't done much design work to tackle this issue yet, all I have at the moment is a vague idea of using a mapping from _something_ to the set of names of the keyword args in the EMF. It doesn't seem to be a single-level 1-to-many mapping, because there may be multiple discriminating types that are used for choosing a method, and the generic-function name itself is not sufficient either.
Maybe I'm just missing something simple. EMFs are compiled, so perhaps the compilation can create the cache/mapping?
On Mon, Oct 12, 2009 at 6:43 PM, Ville Voutilainen ville.voutilainen@gmail.com wrote:
2009/10/12 Erik Huelsmann ehuels@gmail.com:
Profiling one of the performance tests for CL-BENCH which shows huge difference with CLISP, it turned out that our CHECK-INITARGS function in the MOP package is taking 40% of the object instantiation time consumed by MAKE-INSTANCE. After the first invocation, it's possible to replace this function by a lookup in a cache.
I'm not all that surprised that check-initargs shows up with such a high percentage. That's what Peter Graves warned about, being the reason why the checking has been disabled thus far. I've been aware of the need for a cache of some kind, but I'd prefer getting the generic keyword argument checking to a working state before doing optimizations (well, we can call them performance improvements if that sounds nicer :)).
Right. I'm not commenting on the fact that the work done in this area isn't being done. It's more that I found that -conform expectations- we're now impacted big time by argument checking.
We check them pretty well for the methods invoked by make-instance, but that's about it. We also have some other method invocation failures in the ansi-tests.
Right. And while you work on that, I think it's quite ok for me to work on caching the make-instance arguments, right?
I am interested in hearing ideas as to how the cache would be keyed and what exactly would it contain.
Well, ofcourse, there could be serveral options. One would be to have the exact list of options provided and the class supplied and check that for availability in a hash. If it's there, we'd short-cut the return. If it's not, we're actually calculating what answer should be.
Another option is: cache the valid argument options and check the current invocation against the cached options. This requires more computing power and more list traversal upon subsequent calls. However, the total expected cache size has a lower upperbound in the second case than it has in the first case.
I haven't done much design work to tackle this issue yet, all I have at the moment is a vague idea of using a mapping from _something_ to the set of names of the keyword args in the EMF. It doesn't seem to be a single-level 1-to-many mapping, because there may be multiple discriminating types that are used for choosing a method, and the generic-function name itself is not sufficient either.
As we discussed, you cannot expect the solution for the special cases of "make-instance" and the other 4 functions to apply to the GF and its EMFs for the generic case: the CLHS makes explicit exceptions for these methods.
The general GF case can probably be cached within the GF, keyed on the dispatch parameters.
Maybe I'm just missing something simple. EMFs are compiled, so perhaps the compilation can create the cache/mapping?
After compilation, the EMFs are added to the GF using the ADD-METHOD method. i'd say that's about the right time to change/clear caches?
Bye,
Erik.
armedbear-devel@common-lisp.net
-
Erik Huelsmann -
Steve Rolls -
Ville Voutilainen