[mcclim-devel] Severe performance regression with latest CVS sources?
When I update my working copy of McCLIM's CVS sources, I do a quick test to check that things still work fine. I run the CLIM Listener and evaluate a form that draws a monster graph: (time (clim-listener::com-show-class-subclasses t)) Until a few weeks ago, I got run times under 1 s (I seem to remember about 0.7-0.8 s) with CMUCL under Slackware Linux 10.0. With the latest McCLIM CVS sources and CMUCL Snapshot 2006-02 (19C), I now get times over 15 s, no matter whether I initialize multiprocessing with (mp::startup-idle-and-top-level-loops) or not. Can anybody confirm this? Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
test to check that things still work fine. I run the CLIM Listener and evaluate a form that draws a monster graph:
(time (clim-listener::com-show-class-subclasses t))
Until a few weeks ago, I got run times under 1 s (I seem to remember about 0.7-0.8 s) with CMUCL under Slackware Linux 10.0. With the
This refers to a 2.8 GHz Pentium IV machine with 2 GB of RAM. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
Can anybody confirm this?
On my 1100MHz Athlon with 512MiB of RAM, that operation gives the following timing output: Evaluation took: 5.514 seconds of real time 5.418177 seconds of user run time 0.094985 seconds of system run time 0 page faults and 61,853,896 bytes consed. This is with CLIM-desktop and some minor stuff loaded. Are you sure you are not running with more loaded classes now than the last time you ran the test? -- \ Troels "Athas" Henriksen /\ - Insert witty signature
Troels Henriksen <athas@sigkill.dk> writes:
This is with CLIM-desktop and some minor stuff loaded. Are you sure you are not running with more loaded classes now than the last time you ran the test?
For such tests I use a freshly loaded CLIM Listener, with nothing else. Can the additional classes justify an order of magnitude difference in performance? Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Is this with or without Christophe's commit from today of my graph edge patch? I've never looked at the performance impact of that on large graphs. On 4/10/06, Paolo Amoroso <amoroso@mclink.it> wrote:
Troels Henriksen <athas@sigkill.dk> writes:
This is with CLIM-desktop and some minor stuff loaded. Are you sure you are not running with more loaded classes now than the last time you ran the test?
For such tests I use a freshly loaded CLIM Listener, with nothing else. Can the additional classes justify an order of magnitude difference in performance?
Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log _______________________________________________ mcclim-devel mailing list mcclim-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/mcclim-devel
"Andy Hefner" <ahefner@gmail.com> writes:
Is this with or without Christophe's commit from today of my graph edge patch? I've never looked at the performance impact of that on
I have updated my working copy of the CVS sources about 7-8 hours ago. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Hello, Paolo Amoroso writes:
When I update my working copy of McCLIM's CVS sources, I do a quick test to check that things still work fine. I run the CLIM Listener and evaluate a form that draws a monster graph:
(time (clim-listener::com-show-class-subclasses t))
Until a few weeks ago, I got run times under 1 s (I seem to remember about 0.7-0.8 s) with CMUCL under Slackware Linux 10.0. With the latest McCLIM CVS sources and CMUCL Snapshot 2006-02 (19C), I now get times over 15 s, no matter whether I initialize multiprocessing with (mp::startup-idle-and-top-level-loops) or not.
Can anybody confirm this?
Most likely you have run into the same problem I did with Gsharp, namely that output-record creation is now slower with tree-structured output records. You should be able to ask for a sequence output record, or you can do what I did with Gsharp, namely organize the output records into smaller hierarchical units. Take care, -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. ---------------------------------------------------------------------
Robert Strandh <strandh@labri.fr> writes:
Hello,
Paolo Amoroso writes:
When I update my working copy of McCLIM's CVS sources, I do a quick test to check that things still work fine. I run the CLIM Listener and evaluate a form that draws a monster graph:
(time (clim-listener::com-show-class-subclasses t))
Until a few weeks ago, I got run times under 1 s (I seem to remember about 0.7-0.8 s) with CMUCL under Slackware Linux 10.0. With the latest McCLIM CVS sources and CMUCL Snapshot 2006-02 (19C), I now get times over 15 s, no matter whether I initialize multiprocessing with (mp::startup-idle-and-top-level-loops) or not.
Can anybody confirm this?
Most likely you have run into the same problem I did with Gsharp, namely that output-record creation is now slower with tree-structured output records. You should be able to ask for a sequence output record, or you can do what I did with Gsharp, namely organize the output records into smaller hierarchical units.
I would be surprised to find this kind of performance regression for such a simple test (which, furthermore, is probably dominated by the cost of computing the graph layout); I think that Andy and Andreas measured the slowdown from inserting tree output records as a relatively small multiple for about 10000 output records -- not a factor of 20. Admittedly I still haven't optimized the spatial trees for two-dimensions, nor even removed the debugging ordering-constraint-checking code. Be that as it may, rather than speculating further I would suggest to Paolo that he try with the mcclim sources of "a few weeks ago" in the same lisp and compares the results. If there is a difference, then bisecting the cvs checkins should find the offending commit in a small amount of time. Cheers, Christophe
Christophe Rhodes <csr21@cam.ac.uk> writes:
Be that as it may, rather than speculating further I would suggest to Paolo that he try with the mcclim sources of "a few weeks ago" in the same lisp and compares the results. If there is a difference, then
Which CVS tag can I use for accessing such sources? Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
Christophe Rhodes <csr21@cam.ac.uk> writes:
Be that as it may, rather than speculating further I would suggest to Paolo that he try with the mcclim sources of "a few weeks ago" in the same lisp and compares the results. If there is a difference, then
Which CVS tag can I use for accessing such sources?
You can use -D'2006-04-01 23:59 UTC' to access the code present in the repository at, well, 23:59 UTC on the first of April. Generalise to taste :-) Cheers, Christophe
Christophe Rhodes <csr21@cam.ac.uk> writes:
Be that as it may, rather than speculating further I would suggest to Paolo that he try with the mcclim sources of "a few weeks ago" in the same lisp and compares the results. If there is a difference, then
I have checked out a fresh copy of McCLIM's CVS sources with: cvs co -D 2006-03-25 mcclim The test: (time (clim-listener::com-show-class-subclasses t)) takes about 0.92 s on the same system, i.e. a 2.8 GHz Pentium IV machine running Slackware Linux 10.0 and CMUCL Snapshot 2006-02 (19C). Compilation with CMUCL remained broken for about a month before late March 2006. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
Christophe Rhodes <csr21@cam.ac.uk> writes:
Be that as it may, rather than speculating further I would suggest to Paolo that he try with the mcclim sources of "a few weeks ago" in the same lisp and compares the results. If there is a difference, then
I have checked out a fresh copy of McCLIM's CVS sources with:
cvs co -D 2006-03-25 mcclim
The test:
(time (clim-listener::com-show-class-subclasses t))
takes about 0.92 s on the same system, i.e. a 2.8 GHz Pentium IV machine running Slackware Linux 10.0 and CMUCL Snapshot 2006-02 (19C).
Compilation with CMUCL remained broken for about a month before late March 2006.
But that's OK, because your timing means that any performance regression happened /after/ the 25th of March. So you can track down any change between the 25th and now on your system to identify the regressing commit. Cheers, Christophe
Paolo Amoroso <amoroso@mclink.it> writes:
I have checked out a fresh copy of McCLIM's CVS sources with:
cvs co -D 2006-03-25 mcclim
The test:
(time (clim-listener::com-show-class-subclasses t))
takes about 0.92 s on the same system, i.e. a 2.8 GHz Pentium IV
With sources checked out with the -D 2006-04-05 option, I get times around 0.9 s. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
With sources checked out with the -D 2006-04-05 option, I get times around 0.9 s.
I get as less as 0.83 s with 2006-04-08. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
I get as less as 0.83 s with 2006-04-08.
Less than 0.9 s with 2006-04-10. I think there is enough information to identify the relevant changes. Do you need more tests? Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
Less than 0.9 s with 2006-04-10. I think there is enough information to identify the relevant changes. Do you need more tests?
More precisely, as low as 0.75 s (2006-04-10). Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
Paolo Amoroso <amoroso@mclink.it> writes:
I get as less as 0.83 s with 2006-04-08.
Less than 0.9 s with 2006-04-10. I think there is enough information to identify the relevant changes. Do you need more tests?
Well, now we do know what the change was. I can't honestly see why it should slow everything down so much, though; could you profile the CLIMI package and post the top few results? A statistical profile would also be useful if you could get one. (I wonder if the problem is as simple as creating two hash tables per node... surely that can't be such a bottleneck. Could you also find out how many nodes there are in your graph, and time the creation of twice that many hash tables with (make-hash-table)? The other thing to look at is for collisions in your hash tables; check the EDGES-FROM slot of something with a lot of direct subclasses, say STRUCTURE-CLASS or CONDITION or NUMBER, and see if all of the entries are in the same bucket of the hash table) Cheers, Christophe
Christophe Rhodes <csr21@cam.ac.uk> writes:
should slow everything down so much, though; could you profile the CLIMI package and post the top few results? A statistical profile [...]
The message I sent you with this information is awaiting moderator approval because it is about 60 KB long, which is above the allowed size of 40 KB. In the meantime, I'm going to forward it to you offlist. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Paolo Amoroso <amoroso@mclink.it> writes:
Paolo Amoroso <amoroso@mclink.it> writes:
I have checked out a fresh copy of McCLIM's CVS sources with:
cvs co -D 2006-03-25 mcclim
The test:
(time (clim-listener::com-show-class-subclasses t))
takes about 0.92 s on the same system, i.e. a 2.8 GHz Pentium IV
With sources checked out with the -D 2006-04-05 option, I get times around 0.9 s.
Bother. So if you do get times of 15s from HEAD, then it is Andy's bookkeeping / graph edge stuff. Darn. Christophe
participants (5)
-
Andy Hefner -
Christophe Rhodes -
Paolo Amoroso -
Robert Strandh -
Troels Henriksen