[armedbear-devel] Space leak due to threads stuck in LispThread map
Hello all, I'm having some trouble with space leakage that looks to be because the static map in LispThread isn't letting go of completed JavaThreads. Basically what I'm doing on the Java side looks like:
class Foo { public static int main(){ manager = new Manager(); fun = getLispFunction("processResults");
new Thread() { public void run() { LispObject res = generateSomeLisp(); fun.execute(manager, res); }.run(); manager = null; while(1); } }
The lisp side looks like:
(defun processResults (manager res) (let ((*manager* manager)) (declare (special *manager*)) (eval res)))
But the manager never gets destroyed. I used the eclipse Java Memory Analysis tool to track this down to the created JavaThread being referenced as a key in the static map variable in LispThread. Any suggestions on how to get around this? The whole code base is pretty memory intensive so garbage collecting that manager is really a must (and cleaning up the thread is probably a good plan too). Thanks, -Aaron
Hi James, On Thu, Jul 19, 2012 at 10:16 PM, Pendergrass, James A. < James.Pendergrass@jhuapl.edu> wrote:
Hello all, I'm having some trouble with space leakage that looks to be because the static map in LispThread isn't letting go of completed JavaThreads. Basically what I'm doing on the Java side looks like:
class Foo { public static int main(){ manager = new Manager(); fun = getLispFunction("processResults");
new Thread() { public void run() { LispObject res = generateSomeLisp(); fun.execute(manager, res); }.run(); manager = null; while(1); } }
The lisp side looks like:
(defun processResults (manager res) (let ((*manager* manager)) (declare (special *manager*)) (eval res)))
But the manager never gets destroyed. I used the eclipse Java Memory Analysis tool to track this down to the created JavaThread being referenced as a key in the static map variable in LispThread.
Any suggestions on how to get around this? The whole code base is pretty memory intensive so garbage collecting that manager is really a must (and cleaning up the thread is probably a good plan too).
One way around this would be to use the LispThread class from the org.armedbear.lisp package: it removes the thread it creates from the map when the thread.run() function terminates. The other option would be for someone to rewrite the map to use a weak hash table algorithm. The latter is probably the best long term solution anyway.... Bye, Erik.
Hi Erik, Thanks for the response. On Jul 19, 2012, at 6:37 PM, Erik Huelsmann wrote:
Hi James,
On Thu, Jul 19, 2012 at 10:16 PM, Pendergrass, James A. <James.Pendergrass@jhuapl.edu> wrote: Hello all, I'm having some trouble with space leakage that looks to be because the static map in LispThread isn't letting go of completed JavaThreads. Basically what I'm doing on the Java side looks like:
class Foo { public static int main(){ manager = new Manager(); fun = getLispFunction("processResults");
new Thread() { public void run() { LispObject res = generateSomeLisp(); fun.execute(manager, res); }.run(); manager = null; while(1); } }
The lisp side looks like:
(defun processResults (manager res) (let ((*manager* manager)) (declare (special *manager*)) (eval res)))
But the manager never gets destroyed. I used the eclipse Java Memory Analysis tool to track this down to the created JavaThread being referenced as a key in the static map variable in LispThread.
Any suggestions on how to get around this? The whole code base is pretty memory intensive so garbage collecting that manager is really a must (and cleaning up the thread is probably a good plan too).
One way around this would be to use the LispThread class from the org.armedbear.lisp package: it removes the thread it creates from the map when the thread.run() function terminates.
I'm looking in to this, but there seem to be two problems with it: 1) the constructors for LispThread aren't externally visible. 2) even if it was visible the constructor LispThread(Function fun,LispObject name) requires I create a 0-arity closure with the java manager object bound, which always seems to involve LispThread.currentThread() which sticks the current JavaThread in the hash map.
The other option would be for someone to rewrite the map to use a weak hash table algorithm.
The latter is probably the best long term solution anyway....
This does seem like the best long term solution. There seem to be a few WeakConcurrentHashMap implementations around, but I don't see anything in the core jdk (I could easily be missing something; I'm not really a java guy). The choices seem to be Weak or Concurrent but not both. -aaron
Bye,
Erik.
On Jul 20, 2012, at 10:14 AM, Pendergrass, James A. wrote:
Hi Erik, Thanks for the response.
On Jul 19, 2012, at 6:37 PM, Erik Huelsmann wrote:
Hi James,
On Thu, Jul 19, 2012 at 10:16 PM, Pendergrass, James A. <James.Pendergrass@jhuapl.edu> wrote: Hello all, I'm having some trouble with space leakage that looks to be because the static map in LispThread isn't letting go of completed JavaThreads. Basically what I'm doing on the Java side looks like:
class Foo { public static int main(){ manager = new Manager(); fun = getLispFunction("processResults");
new Thread() { public void run() { LispObject res = generateSomeLisp(); fun.execute(manager, res); }.run(); manager = null; while(1); } }
The lisp side looks like:
(defun processResults (manager res) (let ((*manager* manager)) (declare (special *manager*)) (eval res)))
But the manager never gets destroyed. I used the eclipse Java Memory Analysis tool to track this down to the created JavaThread being referenced as a key in the static map variable in LispThread.
Any suggestions on how to get around this? The whole code base is pretty memory intensive so garbage collecting that manager is really a must (and cleaning up the thread is probably a good plan too).
One way around this would be to use the LispThread class from the org.armedbear.lisp package: it removes the thread it creates from the map when the thread.run() function terminates.
I'm looking in to this, but there seem to be two problems with it: 1) the constructors for LispThread aren't externally visible. 2) even if it was visible the constructor LispThread(Function fun,LispObject name) requires I create a 0-arity closure with the java manager object bound, which always seems to involve LispThread.currentThread() which sticks the current JavaThread in the hash map.
The other option would be for someone to rewrite the map to use a weak hash table algorithm.
The latter is probably the best long term solution anyway....
This does seem like the best long term solution. There seem to be a few WeakConcurrentHashMap implementations around, but I don't see anything in the core jdk (I could easily be missing something; I'm not really a java guy). The choices seem to be Weak or Concurrent but not both.
The attached patch is my attempt at modifying LispThread to only hang on to Java Threads using weak references. I haven't tested this extensively, but it solves my problem with out blowing up. -Aaron
-aaron
Bye,
Erik.
_______________________________________________ armedbear-devel mailing list armedbear-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
On 24 July 2012 16:47, Pendergrass, James A. <James.Pendergrass@jhuapl.edu> wrote:
The attached patch is my attempt at modifying LispThread to only hang on to Java Threads using weak references. I haven't tested this extensively, but it solves my problem with out blowing up.
Can you please explain why it checks the thread reference for isAlive(), but performs no reference checks for join()/interrupt()/getStackTrace()?
On Jul 24, 2012, at 2:31 PM, Ville Voutilainen wrote:
On 24 July 2012 16:47, Pendergrass, James A. <James.Pendergrass@jhuapl.edu> wrote:
The attached patch is my attempt at modifying LispThread to only hang on to Java Threads using weak references. I haven't tested this extensively, but it solves my problem with out blowing up.
Can you please explain why it checks the thread reference for isAlive(), but performs no reference checks for join()/interrupt()/getStackTrace()?
Because I didn't have a good answer as to what should be done when calling join(), interrupt(), or getStackTrace() on a LispThread for a Java Thread that no longer exists, so throwing the null pointer exception seemed as valid as anything else. For isAlive() I felt the correct answer was to return false as the LispThread is clearly not alive if the Java Thread is gone. -Aaron
On 26 July 2012 23:21, Pendergrass, James A. <James.Pendergrass@jhuapl.edu> wrote:
Can you please explain why it checks the thread reference for isAlive(), but performs no reference checks for join()/interrupt()/getStackTrace()? Because I didn't have a good answer as to what should be done when calling join(), interrupt(), or getStackTrace() on a LispThread for a Java Thread that no longer exists, so throwing the null pointer exception seemed as valid as anything else. For isAlive() I felt the correct answer was to return false as the LispThread is clearly not alive if the Java Thread is gone. -Aaron
Good enough. :) Thanks.
participants (3)
-
Erik Huelsmann -
Pendergrass, James A. -
Ville Voutilainen