Zach Beane wrote:
On Wed, Feb 20, 2008 at 08:09:18AM -0800, Jeff Cunningham wrote:
I am having a problem that I can't replicate on demand but has been happening with increasing frequency. I am hoping some of you might suggest ways to troubleshoot it. I don't know that it is a Hunchentoot problem, per se, but it may be an interaction problem between Hunchentoot and Apache2 via mod_lisp2.
The symptom is the server is hung first thing in the morning when I check it. The cpu is at 99% activity on the server image (sbcl). When I look in the error_log I see dozens of these:
[snip]
I had trouble like this in the past, and so far in every case it was due to me making a mistake in the handler and going into an infinite loop. I don't know if that's the case for you, but I had good luck debugging it by adding extra info to thread names so I could identify handlers that might be stuck.
http://xach.livejournal.com/132391.html has a writeup.
You can also troubleshoot by interrupting all threads and requesting their backtrace (best to write it to a file or save them to a table or something like that). That can also help identify problems.
Zach _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
Thanks, Zach. I just tried the following:
(mapcar (lambda (thread) (sb-thread:interrupt-thread thread (lambda () (throw 'hunchentoot::handler-done "")))) (sb-thread:list-all-threads))
And it says it can't find the tag 'hunchentoot::handler-done (I assume). Yet, it shows up in the list of package symbols, exported even:
attempt to THROW to a tag that does not exist: HANDLER-DONE [Condition of type SB-INT:SIMPLE-CONTROL-ERROR]
Restarts: 0: [ABORT] Abort SLIME compilation. 1: [ABORT] Return to SLIME's top level. 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "worker" {D813B61}>)
Backtrace: 0: ("no debug information for frame") 1: ((LAMBDA NIL)) 2: (SB-THREAD:INTERRUPT-THREAD #<SB-THREAD:THREAD "worker" {D813B61}> #<FUNCTION (LAMBDA NIL) {D84CC55}>) 3: (SB-IMPL::MAP1 #<FUNCTION (LAMBDA (THREAD)) {D84CF95}> ((#<SB-THREAD:THREAD "worker" {D813B61}> #<SB-THREAD:THREAD "hunchentoot-worker-34" {D7CB2D1}> #<SB-THREAD:THREAD "hunchentoot-worker-33" {D7D92B9}> #<SB-THREAD:THREAD "hunchentoot-worker-32" {D777351}> #<SB-THREAD:THREAD "hunchentoot-worker-31" {DA38F11}> #<SB-THREAD:THREAD "hunchentoot-worker-30" {DC5A989}> ...)) :LIST T) 4: (MAPCAR #<FUNCTION (LAMBDA (THREAD)) {D84CF95}> (#<SB-THREAD:THREAD "worker" {D813B61}> #<SB-THREAD:THREAD "hunchentoot-worker-34" {D7CB2D1}> #<SB-THREAD:THREAD "hunchentoot-worker-33" {D7D92B9}> #<SB-THREAD:THREAD "hunchentoot-worker-32" {D777351}> #<SB-THREAD:THREAD "hunchentoot-worker-31" {DA38F11}> #<SB-THREAD:THREAD "hunchentoot-worker-30" {DC5A989}> ...))
--Jeff