[cxml-devel] CXML performance
Hi, I tried using CXML to parse a slightly big (~1.2 MB) XML in ACL 8.0. I am using the following form to parse it into a DOM: (cxml:parse xml (cxml-dom:make-dom-builder)) ; xml is a string Generally, it takes about 0.73-0.75 sec to parse on my machine. What also happens is that, while the parser is running, it blocks all the other threads i.e. they become unresponsive. Any clues on why it can be happening? And how it can be fixed? Also, ACL on OS X (and Linux) does not run any Lisp code while a foreign call is being made (thus blocking all the other threads). Is there any chance that CXML calls any foreign code? Chaitanya
On Thu, 04 Sep 2008 20:23:04 +0530, Chaitanya Gupta <mail@chaitanyagupta.com> wrote:
I tried using CXML to parse a slightly big (~1.2 MB) XML in ACL 8.0. I am using the following form to parse it into a DOM:
(cxml:parse xml (cxml-dom:make-dom-builder)) ; xml is a string
Generally, it takes about 0.73-0.75 sec to parse on my machine. What also happens is that, while the parser is running, it blocks all the other threads i.e. they become unresponsive. Any clues on why it can be happening? And how it can be fixed?
Also, ACL on OS X (and Linux) does not run any Lisp code while a foreign call is being made (thus blocking all the other threads). Is there any chance that CXML calls any foreign code?
Unless they've changed this recently, AllegroCL (like LispWorks pre-6.x) has a big lock which blocks all other threads no matter whether they're Lisp or foreign threads. Edi.
Edi Weitz wrote:
Unless they've changed this recently, AllegroCL (like LispWorks pre-6.x) has a big lock which blocks all other threads no matter whether they're Lisp or foreign threads.
From Franz: <quote> In some senses, his statement is correct, but not because of any "big lock". The effect is seen on "virtual" or "green" threads implementations like the OS X/Linux versions you are discussing in that thread. Note that our virtual threads implementation uses only one thread, and so if you have another real thread (spawned by pthread_create or similar) the lisp will have no effect on that thread. If you spawn the thread through lisp, via e.g. process-run-function, then it is not really a "thread", because it is virtual, and has to wait for the (virtual) scheduler to schedule it. Other real threads will run as the system sees fit. On Windows, we have an implementation called os-threads which ties the thread spawned by process-run-function directly to a real thread, and so it acts precisely as you implied it would in the cxml-devel mail thread. </quote> Chaitanya
On Tue, 09 Sep 2008 12:57:38 +0530, Chaitanya Gupta <mail@chaitanyagupta.com> wrote:
<quote> In some senses, his statement is correct, but not because of any "big lock". The effect is seen on "virtual" or "green" threads implementations like the OS X/Linux versions you are discussing in that thread. Note that our virtual threads implementation uses only one thread, and so if you have another real thread (spawned by pthread_create or similar) the lisp will have no effect on that thread. If you spawn the thread through lisp, via e.g. process-run-function, then it is not really a "thread", because it is virtual, and has to wait for the (virtual) scheduler to schedule it. Other real threads will run as the system sees fit.
On Windows, we have an implementation called os-threads which ties the thread spawned by process-run-function directly to a real thread, and so it acts precisely as you implied it would in the cxml-devel mail thread. </quote>
Call it "big lock" or not, for me the actual question is whether two Lisp threads in the same imagine can actually run /at the same time/ if I have a multi-core or multi-processor system. My understanding is that they can't. Edi.
Hi, Quoting Edi Weitz (edi@agharta.de):
On Tue, 09 Sep 2008 12:57:38 +0530, Chaitanya Gupta <mail@chaitanyagupta.com> wrote:
<quote> In some senses, his statement is correct, but not because of any "big lock". The effect is seen on "virtual" or "green" threads
[...]
Call it "big lock" or not, for me the actual question is whether two Lisp threads in the same imagine can actually run /at the same time/ if I have a multi-core or multi-processor system. My understanding is that they can't.
this discussion is all very interesting, but not really related to cxml anymore. Perhaps you could take it elsewhere? Thanks, d.
Hi, Quoting Chaitanya Gupta (mail@chaitanyagupta.com):
I tried using CXML to parse a slightly big (~1.2 MB) XML in ACL 8.0. I am using the following form to parse it into a DOM:
(cxml:parse xml (cxml-dom:make-dom-builder)) ; xml is a string
Generally, it takes about 0.73-0.75 sec to parse on my machine.
I'd accept that for now. You could try (cxml:parse xml nil) to see how much time XML parsing itself takes (as opposed to DOM building). I don't really have a recommendation regarding speed improvements either way though.
What also happens is that, while the parser is running, it blocks all the other threads i.e. they become unresponsive. Any clues on why it can be happening? And how it can be fixed?
Sounds to me like an artifact of ACL's scheduling algorithm. On my version of ACL, the variable mp:*default-process-quantum* defaults to 2. If I understand correctly, this is measured in seconds, so if you just had a context switch before calling CXML:PARSE, which takes 0.75s in your case, ACL won't even consider giving time to any other thread. (In particular because you're parsing from a string, so cxml doesn't spend any time blocking on input. The situation with files already buffered by the kernel might be similar though.) You could try setting a lower process quantum if this bothers you.
Also, ACL on OS X (and Linux) does not run any Lisp code while a foreign call is being made (thus blocking all the other threads). Is there any chance that CXML calls any foreign code?
No, I don't think so. d.
participants (3)
-
Chaitanya Gupta -
David Lichteblau -
Edi Weitz