- armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] Thread safety of the compiler (was: Thread safety of clos.lisp)
by Erik Huelsmann 17 Jul '09

17 Jul '09

Well, I've been investigating the issue with our clos.lisp not being thread safe and me seeing NullPointerExceptions. It turns out that GENSYM was not thread safe in case there were no thread-local special bindings: the SYMBOL-VALUE wasn't protected against being manipulated from 2 threads at the same time. I changed that. Now, even though clos.lisp is not thread safe, I'm not seeing any NullPointerExceptions anymore. The NullPointerExceptions were caused by the fact that COMPILE was running in 2 threads at the same time, generating the same symbol at the same time. Then, it called REMEMBER from both threads, one overwriting the value stored by the other. That doesn't cause NullPointerExceptions by itself, but when each thread loads its COMPILEd function, the constructor calls Lisp.recall(), which removes the key from the hash used for temp-storage. The second function then finds 'null' instead of the value it stored. So, that's another mystery solved. Fixed in trunk now. However, what remains is the fact that the remember/recall combo uses global keys while COMPILE may be generating thread-local key values. I'm wondering what to do about that. Possibly I should just create a ThreadLocal which has a temp-storage hash table per thread and document remember/recall are a thread-local storage mechanism. Of course, your opinions are welcome as always, especially with respect to the last paragraph. Bye, Erik.

1 1

[armedbear-devel] Thread safety of clos.lisp (and supporting Java classes)
by Erik Huelsmann 15 Jul '09

15 Jul '09

During my testing of multi-threaded use of ABCL over the past weeks, I found that ABCL performs reliably with 2 threads on 2 cores (fully dedicated to running ABCL code). However, there's one notable exception: Any code which requires CLOS, such as the pretty printer and other actions which require generic methods. I'm able to quite reliably break ABCL by starting a specific series of actions on 2 threads at the same time. (No global values in the program code!) The cause can be led back to slow-method-lookup; if I add SYNCHRONIZED-ON to it, the tests work fine. However, SLOW-METHOD-LOOKUP has friends which also manipulate the EMF-CACHE, so I don't think this would be the full solution. There's however an additional issue: Hashmap is defined to be thread-unsafe: 2 concurrent readers are fine, but concurrent writers or mixed concurrent readers/writers are not guaranteed to work. So, to guarantee results with the current EMF-CACHE implementation, we need to make sure there's 1 reader/writer at the same time - using synchronized-on, that is. For most applications this would be horribly inefficient: most methods will be added once and read huge numbers of times. ECL chose a rather unconventional direction in this case: instead of searching for lock-less semantics of global state, they have an EMF-CACHE per thread, meaning that read/write operations on the cache are thread-local and hence can be done lockless. ABCL could easily go with java.util.concurrent.ConcurrentHashMap, which is lock-less and allows concurrent readers and limited synchronous writers (~16 by default). However, this assumes that the actual threading issue is mainly with writing values to the cache. ie, before an "entry to be cached" is available, nothing happens which can't be overwritten immediately after it has been added to the cache. It's that last bit I have problems with assessing: do we simply need to use another caching hash class or do we need to do much broader "global" locking? (The "global" locking proposed would simply be locking that specific generic function; clos could easily run in other threads for other GFs...) Help welcome! Bye, Erik.

1 0

[armedbear-devel] Compiler macros vs source transforms
by Erik Huelsmann 14 Jul '09

14 Jul '09

While reading through precompiler.lisp, I noticed we have 3 concepts for transformation of lisp sources * normal macros, * compiler-macros, * source-transforms The difference between macros and compiler-macros is well documented in the CLHS; however, the difference between compiler-macros and source-transforms is unclear to me. I found that the former is expanded in compiler-pass1 while the latter is expanded in precompiler.lisp. However, when we're compiling for interpretation (not compilation), *in-jvm-compile* is NIL, meaning that source-transforms don't get expanded. The net effect would be that both source-transforms and compiler-macros only get expanded when we're compiling. I'm wondering what the benefit of source transforms is if this is really true. Bye, Erik.

3 4

[armedbear-devel] LispThread.execute, funcall(), stack frames
by Tobias C. Rittweiler 14 Jul '09

14 Jul '09

I just added http://trac.common-lisp.net/armedbear/ticket/60 Essentially the code needs some refactoring. It's not obvious that LispThread.execute is "execute with stack frame". It's also not really obvious when to use it. In fact, why is it there? Isn't it the same as Lisp.funcall()? -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

2 1

[armedbear-devel] patch: do not call compile for every call of eql-specialized generic functions
by Anton Vodonosov 14 Jul '09

14 Jul '09

Hello. The attached is a patch to improve performance of eql-specialized generic functions. I started to investigate this problem because slime fuzzy completion works very slow with ABCL. For example (time (swank::fuzzy-completions "de" 'cl-user)) takes 1.5 - 2 seconds. That long time is not pleasant for user interface. Turned out that most of the time is spent in COMPILE. During SWANK::FUZZY-COMPLETIONS every symbol is processed by SWANK-CLASSIFY-SYMBOL, which uses (DOCUMENTATION s 'TYPE), and every invocation (DOCUMENTATION s 'TYPE) being eql-specialized function leads to COMPILE. With my patch (time (swank::fuzzy-completions "de" 'cl-user)) takes 0.25 - 0.5 seconds. Not too fast, in CLISP the same takes 0.07 sec, in CCL 0.03 sec. But significantly better than it was. In the patch method getArgSpecialization could probably be named better. Also I am not sure if you want to keep the wordy javadoc comment; i just decided to put the patch explanation to the sources instead of this email. Best regards, - Anton

2 3

[armedbear-devel] [PATCH] build-from-lisp.sh (take 2)
by Tobias C. Rittweiler 14 Jul '09

14 Jul '09

Ok, I've redone build-from-lisp.sh so that build-abcl.lisp can remain untouched. I also added the feature to set :clean, :full, :batch from the command-line via --clean, --full, and --batch. I used the script to build ABCL successfully on SBCL, CLISP, and CCL. -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

2 1

[armedbear-devel] [PATCH] fix arglist of EXT:ARGLIST
by Tobias C. Rittweiler 13 Jul '09

13 Jul '09

Fix (ext:arglist 'ext:arglist) => NIL. -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

2 1

[armedbear-devel] [PATCH] build-from-lisp.sh
by Tobias C. Rittweiler 13 Jul '09

13 Jul '09

The following makes it possible to build ABCL from the command line directly: ./build-from-lisp.sh sbcl ./build-from-lisp.sh clisp etc. -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

3 3

[armedbear-devel] [PATCH] Move some symbols into the THREADS package.
by Tobias C. Rittweiler 13 Jul '09

13 Jul '09

The patch attached to this mail moves the usual suspects to the THREADS package. It does not move the Gates-related functions because I think that Gates can be removed again now with the SYNCHRONIZE-ON stuff. -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

2 1

[armedbear-devel] gmane.lisp.armedbear.devel
by Tobias C. Rittweiler 13 Jul '09

13 Jul '09

With this message, the gmane.lisp.armedbear.devel group should be created. -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

1 0