Hi all,
I recently started playing with LispWorks, Apache w/ mod_lisp2 and TBNL. I'm using the newest TBNL as well as the newest versions of all its dependencies. I'm using the free version of LispWorks and therefore have a heap size limit, and I'm using Windows XP Pro.
The problem was that I after I got everything working I was randomly clicking on the test page link when LispWorks quickly died after a few simple clicks. It seemed to leak 3-4 megs per click!
Upon further investigation it turns out this is a Firefox specific problem (haven't tested Opera yet :) ) and it doesn't happen in IE. Checking the Apache logs I get lots of these when browsing with Firefox:
[Thu Sep 29 19:08:10 2005] [notice] [TBNL] No session for session identifier '1:158C42649C449770460538C62DC7A0EF' (User-Agent: 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7', IP: '127.0.0.1')
So obviously there is a session handling problem with Firefox? I'm guessing that it's starting a new session with each refresh, and hence the memory usage.
I'll look into it further and try to get into some of the TBNL code, but I'm a rookie with Lisp so it might take me a while! Any help would be appreciated...
Thanks!
Anthony
Hi,
I've seen this before (long ago) and can't quite remember what caused it. I believe it has to do with restarting the lisp/TBNL server without restarting apache. Try cleaning Firefox's cache, if that doesn't work try cleaning everything in Firefox (the 'clear all' button in preference>privacy). You might have to refresh a couple of times too. It is a real pain.
If I remember, this is one of the reasons I work day-to-day with a stand-alone version of TBNL (either through Araneida or TBNL's built in capabilities (in fact, this is, I think, the reason I contributed the TBNL stand-alone stuff in the first place :-))
Cheers, Bob
On Sep 29, 2005, at 10:25 PM, Anthony F wrote:
Hi all,
I recently started playing with LispWorks, Apache w/ mod_lisp2 and TBNL. I'm using the newest TBNL as well as the newest versions of all its dependencies. I'm using the free version of LispWorks and therefore have a heap size limit, and I'm using Windows XP Pro.
The problem was that I after I got everything working I was randomly clicking on the test page link when LispWorks quickly died after a few simple clicks. It seemed to leak 3-4 megs per click!
Upon further investigation it turns out this is a Firefox specific problem (haven't tested Opera yet :) ) and it doesn't happen in IE. Checking the Apache logs I get lots of these when browsing with Firefox:
[Thu Sep 29 19:08:10 2005] [notice] [TBNL] No session for session identifier '1:158C42649C449770460538C62DC7A0EF' (User-Agent: 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/ 20050915 Firefox/1.0.7', IP: '127.0.0.1')
So obviously there is a session handling problem with Firefox? I'm guessing that it's starting a new session with each refresh, and hence the memory usage.
I'll look into it further and try to get into some of the TBNL code, but I'm a rookie with Lisp so it might take me a while! Any help would be appreciated...
Thanks!
Anthony _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
---- Bob Hutchison -- blogs at http://www.recursive.ca/hutch/ Recursive Design Inc. -- http://www.recursive.ca/ Raconteur -- http://www.raconteur.info/
On Sep 30, 2005, at 8:59 AM, Bob Hutchison wrote:
Hi,
I've seen this before (long ago) and can't quite remember what caused it. I believe it has to do with restarting the lisp/TBNL server without restarting apache. Try cleaning Firefox's cache, if that doesn't work try cleaning everything in Firefox (the 'clear all' button in preference>privacy). You might have to refresh a couple of times too. It is a real pain.
And you know, thinking about it, I think I had to restart Apache too.
If I remember, this is one of the reasons I work day-to-day with a stand-alone version of TBNL (either through Araneida or TBNL's built in capabilities (in fact, this is, I think, the reason I contributed the TBNL stand-alone stuff in the first place :-))
Cheers, Bob
On Sep 29, 2005, at 10:25 PM, Anthony F wrote:
Hi all,
I recently started playing with LispWorks, Apache w/ mod_lisp2 and TBNL. I'm using the newest TBNL as well as the newest versions of all its dependencies. I'm using the free version of LispWorks and therefore have a heap size limit, and I'm using Windows XP Pro.
The problem was that I after I got everything working I was randomly clicking on the test page link when LispWorks quickly died after a few simple clicks. It seemed to leak 3-4 megs per click!
Upon further investigation it turns out this is a Firefox specific problem (haven't tested Opera yet :) ) and it doesn't happen in IE. Checking the Apache logs I get lots of these when browsing with Firefox:
[Thu Sep 29 19:08:10 2005] [notice] [TBNL] No session for session identifier '1:158C42649C449770460538C62DC7A0EF' (User-Agent: 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/ 20050915 Firefox/1.0.7', IP: '127.0.0.1')
So obviously there is a session handling problem with Firefox? I'm guessing that it's starting a new session with each refresh, and hence the memory usage.
I'll look into it further and try to get into some of the TBNL code, but I'm a rookie with Lisp so it might take me a while! Any help would be appreciated...
Thanks!
Anthony _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
Bob Hutchison -- blogs at http://www.recursive.ca/hutch/ Recursive Design Inc. -- http://www.recursive.ca/ Raconteur -- http://www.raconteur.info/
---- Bob Hutchison -- blogs at http://www.recursive.ca/hutch/ Recursive Design Inc. -- http://www.recursive.ca/ Raconteur -- http://www.raconteur.info/
On Fri, 30 Sep 2005 09:01:01 -0400, Bob Hutchison hutch@recursive.ca wrote:
And you know, thinking about it, I think I had to restart Apache too.
Yeah, that's one of the drawbacks of the mod_lisp backend - during development there are situations where you /must/ restart Apache or you have to cope with dangling connections. I'm glad you contributed the stand-alone backend, that's what I'm using for development now... :)
To the OP: Firefox/Win is my main browser and LispWorks is my main Lisp platform. I'm not aware of any problems specific to these. What exactly do you mean by "seemed to leak 3-4 megs per click?"
As for the session stuff in the logs you might want to check with something like "Live HTTP Headers" to see what's going on.
Cheers, Edi.
Perhaps "Leak" was too strong a word. I'm not sure exactly what is going on yet, but here are the symptoms:
I start Apache, then TBNL. I hit the test page with Firefox. While watching the Process tab in Task Manager I follow a link. Memory usage on the LispWorks process jumps anywhere from 2 to 5 megs. Following any other links within the test app or refreshing the page causes the same until I hit LispWorks Personal's heap size limit and it dies.
I have yet to check if restarting Apache fixes the problem, but I will report on that later. HOWEVER, I notice that even with IE a very small amount of memory seems to be leaking? If you hit the Test app from IE, follow a link and/or repeatedly refresh I notice memory usage on the LispWorks process creep upwards ever so slowly.
Having said this, I have not actually done any real investigating yet (like I said, I'm a total newb)... so this may or may not be a result of expected behavior.
On a seperate note: I don't really want to start a mod_lisp vs. other options war, but is anyone using a Lisp server (AllegroServer, TBNL's server, etc) back end in a production environment? I was wondering about performance, security, other issues, etc. Personally, I feel "safer" using Apache as it's battle hardened and fairly ubiquitous, but maybe those feelings are unfounded? Any comments?
Thanks!
Anthony
----- Original Message ----- From: "Edi Weitz" edi@agharta.de To: "Bob Hutchison" hutch@recursive.ca Cc: tbnl-devel@common-lisp.net Sent: Friday, September 30, 2005 6:26 AM Subject: Re: [tbnl-devel] Firefox
On Fri, 30 Sep 2005 09:01:01 -0400, Bob Hutchison hutch@recursive.ca wrote:
And you know, thinking about it, I think I had to restart Apache too.
Yeah, that's one of the drawbacks of the mod_lisp backend - during development there are situations where you /must/ restart Apache or you have to cope with dangling connections. I'm glad you contributed the stand-alone backend, that's what I'm using for development now... :)
To the OP: Firefox/Win is my main browser and LispWorks is my main Lisp platform. I'm not aware of any problems specific to these. What exactly do you mean by "seemed to leak 3-4 megs per click?"
As for the session stuff in the logs you might want to check with something like "Live HTTP Headers" to see what's going on.
Cheers, Edi. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
On Sep 30, 2005, at 2:08 PM, Anthony F wrote:
Perhaps "Leak" was too strong a word. I'm not sure exactly what is going on yet, but here are the symptoms:
I start Apache, then TBNL. I hit the test page with Firefox. While watching the Process tab in Task Manager I follow a link. Memory usage on the LispWorks process jumps anywhere from 2 to 5 megs. Following any other links within the test app or refreshing the page causes the same until I hit LispWorks Personal's heap size limit and it dies.
I have yet to check if restarting Apache fixes the problem, but I will report on that later. HOWEVER, I notice that even with IE a very small amount of memory seems to be leaking? If you hit the Test app from IE, follow a link and/or repeatedly refresh I notice memory usage on the LispWorks process creep upwards ever so slowly.
Hmm, I do not experience this with my application at all. Now, I do know that it takes a while for memory usage in LW to settle down. You might want to try automating it with wget+script or ab and see what happens. (ab is apache's benchmarking tool, and that's where the name comes from)
Now, under OS/X using TBNL standalone I do get an infinite growth in memory that will eventually kill LW -- eventually is within a few seconds (or if I'm using ab, a small fraction of a second). This doesn't happen under linux (by personal testing) or windows (by report). It also doesn't happen using Araneida or Apache on OS/X. I mention this because you might have somehow encountered this problem with a different configuration. Unfortunately, I have never worked out what the problem is.
Having said this, I have not actually done any real investigating yet (like I said, I'm a total newb)... so this may or may not be a result of expected behavior.
On a seperate note: I don't really want to start a mod_lisp vs. other options war, but is anyone using a Lisp server (AllegroServer, TBNL's server, etc) back end in a production environment? I was wondering about performance, security, other issues, etc. Personally, I feel "safer" using Apache as it's battle hardened and fairly ubiquitous, but maybe those feelings are unfounded? Any comments?
I've been running a demo version of the system I've been working on, stand-alone TBNL on linux, light load, but running continuously since the first week of June. Performance of TBNL without Apache in front of it (either Araneida or standalone) is phenomenal -- I've got a Java background and this level of performance is hard to get used to :-) Apache slows this down by pretty much exactly a factor of two on OS/X, but that is still an embarrassment for Java -- no Java server that I've used (tomcat, jetty, jboss, weblogic, websphere, resin, jrun) can do anything like this. Edi's done a wonderful job with TBNL -- fast and easy to use.
Cheers, Bob
Thanks!
Anthony
----- Original Message ----- From: "Edi Weitz" edi@agharta.de To: "Bob Hutchison" hutch@recursive.ca Cc: tbnl-devel@common-lisp.net Sent: Friday, September 30, 2005 6:26 AM Subject: Re: [tbnl-devel] Firefox
On Fri, 30 Sep 2005 09:01:01 -0400, Bob Hutchison hutch@recursive.ca wrote:
And you know, thinking about it, I think I had to restart Apache too.
Yeah, that's one of the drawbacks of the mod_lisp backend - during development there are situations where you /must/ restart Apache or you have to cope with dangling connections. I'm glad you contributed the stand-alone backend, that's what I'm using for development now... :)
To the OP: Firefox/Win is my main browser and LispWorks is my main Lisp platform. I'm not aware of any problems specific to these. What exactly do you mean by "seemed to leak 3-4 megs per click?"
As for the session stuff in the logs you might want to check with something like "Live HTTP Headers" to see what's going on.
Cheers, Edi. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
---- Bob Hutchison -- blogs at http://www.recursive.ca/hutch/ Recursive Design Inc. -- http://www.recursive.ca/ Raconteur -- http://www.raconteur.info/
On Fri, 30 Sep 2005 11:08:09 -0700, "Anthony F" eatme444@hotmail.com wrote:
Perhaps "Leak" was too strong a word. I'm not sure exactly what is going on yet, but here are the symptoms:
I start Apache, then TBNL. I hit the test page with Firefox. While watching the Process tab in Task Manager I follow a link. Memory usage on the LispWorks process jumps anywhere from 2 to 5 megs. Following any other links within the test app or refreshing the page causes the same until I hit LispWorks Personal's heap size limit and it dies.
I have yet to check if restarting Apache fixes the problem, but I will report on that later. HOWEVER, I notice that even with IE a very small amount of memory seems to be leaking? If you hit the Test app from IE, follow a link and/or repeatedly refresh I notice memory usage on the LispWorks process creep upwards ever so slowly.
Having said this, I have not actually done any real investigating yet (like I said, I'm a total newb)... so this may or may not be a result of expected behavior.
I've just experimented with this myself a little bit and I have to admit that TBNL on LispWorks does indeed seem to "leak" memory in that the size of the LW application grows continuously without shrinking again - on Linux as well as on Windows. It turns out that a lot of this is due to the external format code I introduced in 0.8.0 which uses the FLI. In particular, there's a call to SYS:IN-STATIC-AREA in there which I guess is the culprit. It was my understanding that these statically allocated objects are subject to normal garbage collection but it looks as if new static segments are allocated all the time. As the LW reference manual says: "Such additional segments would cause the application to grow irreversibly."
But even pre-0.8.0 TBNL on LW "leaks" a bit. It looks like at least under heavy load the higher generations tend to grow so one should probably call MARK-AND-SWEEP from time to time.
I'm a bit too busy right now to investigate this further but maybe someone else wants to look at it and fix it.
On a seperate note: I don't really want to start a mod_lisp vs. other options war, but is anyone using a Lisp server (AllegroServer, TBNL's server, etc) back end in a production environment? I was wondering about performance, security, other issues, etc. Personally, I feel "safer" using Apache as it's battle hardened and fairly ubiquitous, but maybe those feelings are unfounded? Any comments?
I don't think there's a reason to assume that Lisp servers are inherently less secure than Apache, actually I tend to think that there are a lot of "typical" security holes that you'll see in a C/C++ app but not in a Lisp app.
As fas as performance is concerned I'm pretty sure that Lisp servers are definitely "fast enough." If one of the websites I'm maintaining should really ever have so much traffic that it hits the performance limits one can see when benchmarking Apache/mod_lisp/TBNL I'd be very happy... :)
Having said that - there aren't many Lisp web servers out there that fully implement HTTP/1.1 and you certainly want that if you're serving production web sites. TBNL stand-alone doesn't do HTTP/1.1 and Araneida doesn't do it either AFAIK. AllegroServer does, though, and it has been used successfully in production environments like these:
http://www.pandorabots.com/ http://www.franz.com/
Another example for a Lisp web server used in a production environment, although a bit dated, is the White House Electronic Publications server which was running atop CL-HTTP when Clinton was still in office. CL-HTTP is also HTTP/1.1-compliant, BTW. It just has a somewhat peculiar license.
Cheers, Edi.
On Sun, 02 Oct 2005 03:26:11 +0200, Edi Weitz edi@agharta.de wrote:
I've just experimented with this myself a little bit and I have to admit that TBNL on LispWorks does indeed seem to "leak" memory in that the size of the LW application grows continuously without shrinking again - on Linux as well as on Windows. It turns out that a lot of this is due to the external format code I introduced in 0.8.0 which uses the FLI. In particular, there's a call to SYS:IN-STATIC-AREA in there which I guess is the culprit.
Yes, that /was/ the culprit. It turns out this is a bug in LW which'll be fixed soon:
http://thread.gmane.org/gmane.lisp.lispworks.general/4604
Until then there's no immediate solution other than using undocumented functions like ENCODE-LISP-STRING which don't work well with delivery.
Cheers, Edi.
On Tue, 04 Oct 2005 20:40:56 +0200, Edi Weitz edi@agharta.de wrote:
Yes, that /was/ the culprit. It turns out this is a bug in LW which'll be fixed soon:
http://thread.gmane.org/gmane.lisp.lispworks.general/4604
Until then there's no immediate solution other than using undocumented functions like ENCODE-LISP-STRING which don't work well with delivery.
OK, thanks to LispWorks support there is a better solution now. If you're using LispWorks 4.4.5 on Windows or Linux (/not/ on OS X as the problem doesn't happen there) you can send an email to
lisp-support@lispworks.com
asking for the patch SYSTEM-CONS-FREE-CHAIN which you'll get for free. Please make sure to use the "Report Bug" command from the LW IDE to include a full bug report form.
The new 0.8.3 release is just intended to alert users who don't read this mailing list of this problem.
ChangeLog:
Version 0.8.3 2005-10-10 Alert LW users that a patch for OCTETS-TO-STRINGS is available (thanks to LispWorks support)
Download:
http://weitz.de/files/tbnl.tar.gz
Cheers, Edi.
-
Anthony F -
Bob Hutchison -
Edi Weitz