Since we seem to have lots of issues with our Trac install (most people find it slow; some find its availability disappointing), I thought I'd look at the Trac website to find out how their dev team advises on these matters. Well, the conclusion is simple: we're using the 'tracd' method of running Trac and this is certainly the only way how *not* to do it.
They advise CGI for small sites or FastCGI/WSGI for larger sites. I estimate us to be in the latter category, so I took my own VPS (which is in the former category, but also runs Trac) and configured it for FastCGI. The steps turn out to be quite simple. I volunteer to make the same changes on c-l.net (which drewc already set me up for), but I'm posting here for review. The last thing I'd want is to open up a security hole in the site.
Steps are:
* Install mod_fcgid (although I used mod_fastcgi, which is older and not recommended by the Trac team; it works though) * Add the script handlers and the script alias to the httpd.conf file within the virtual host section for trac.c-l.net:
AddHandler fastcgi-script .fcgi ScriptAlias / /path/to/apache/cgi-bin/trac.fcgi/
* copy the 'web/' subdir of the installed Trac into the cgi-bin directory * copy in the cgi-bin directory "fcgi_frontend.py" to "trac.fcgi" * add to the trac.fcgi script - before the first 'import' statement:
import os os.environ['PYTHON_EGG_CACHE'] = "/tmp/httpd-egg-cache" os.environ['TRAC_ENV_PARENT_DIR'] = "/path/to/parent/trac/project/dirs"
* create /tmp/httpd-egg-cache with the user and group used to run httpd and '700' access settings * restart apache
Comments?
The effect of the change is this: by default, a single trac.fcgi application will be loaded - just as with our current situation. However, on high demand, additional instances will be spawned - contrary to the current situation, where a single tracd will try to handle all requests. When the load is over, the number of servers is scaled back to normal (1).
Bye,
Erik.
Erik,
please go ahead and make the change. I'm not sure if moving to FastCGI will really solve the problems, though, as the root issue seems to be that we're seeing robots that request pages which create a lot of load to generate. In any case, moving away from tracd is TRT, and we can then try to improve things further by using a caching proxy like squid.
Whatever you do, please use the Debian packages if at all possible so that we have a chance to upgrade.
Thanks, Hans
On Sun, Mar 7, 2010 at 22:27, Erik Huelsmann ehuels@gmail.com wrote:
Since we seem to have lots of issues with our Trac install (most people find it slow; some find its availability disappointing), I thought I'd look at the Trac website to find out how their dev team advises on these matters. Well, the conclusion is simple: we're using the 'tracd' method of running Trac and this is certainly the only way how *not* to do it.
They advise CGI for small sites or FastCGI/WSGI for larger sites. I estimate us to be in the latter category, so I took my own VPS (which is in the former category, but also runs Trac) and configured it for FastCGI. The steps turn out to be quite simple. I volunteer to make the same changes on c-l.net (which drewc already set me up for), but I'm posting here for review. The last thing I'd want is to open up a security hole in the site.
Steps are:
* Install mod_fcgid (although I used mod_fastcgi, which is older and not recommended by the Trac team; it works though) * Add the script handlers and the script alias to the httpd.conf file within the virtual host section for trac.c-l.net:
AddHandler fastcgi-script .fcgi ScriptAlias / /path/to/apache/cgi-bin/trac.fcgi/
* copy the 'web/' subdir of the installed Trac into the cgi-bin directory * copy in the cgi-bin directory "fcgi_frontend.py" to "trac.fcgi" * add to the trac.fcgi script - before the first 'import' statement:
import os os.environ['PYTHON_EGG_CACHE'] = "/tmp/httpd-egg-cache" os.environ['TRAC_ENV_PARENT_DIR'] = "/path/to/parent/trac/project/dirs"
* create /tmp/httpd-egg-cache with the user and group used to run httpd and '700' access settings * restart apache
Comments?
The effect of the change is this: by default, a single trac.fcgi application will be loaded - just as with our current situation. However, on high demand, additional instances will be spawned - contrary to the current situation, where a single tracd will try to handle all requests. When the load is over, the number of servers is scaled back to normal (1).
Bye,
Erik.
clo-devel mailing list clo-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/clo-devel
@Erik : please go ahead, sounds great.
@Hans : We've been running behind squid for 1/2 a year now... it helped at first, but the real issue is with dynamic pages, and tracd does not seem to set proper cache control headers for things that can't/don't change.
Also, tracd does not allow a robots.txt, so FCGI will allow us to cut off yahoo and msnbot, who are the real trouble IMO.
Cheers,
drewc
On 7 March 2010 21:41, Hans Hübner hans.huebner@gmail.com wrote:
Erik,
please go ahead and make the change. I'm not sure if moving to FastCGI will really solve the problems, though, as the root issue seems to be that we're seeing robots that request pages which create a lot of load to generate. In any case, moving away from tracd is TRT, and we can then try to improve things further by using a caching proxy like squid.
Whatever you do, please use the Debian packages if at all possible so that we have a chance to upgrade.
Thanks, Hans
On Sun, Mar 7, 2010 at 22:27, Erik Huelsmann ehuels@gmail.com wrote:
Since we seem to have lots of issues with our Trac install (most people find it slow; some find its availability disappointing), I thought I'd look at the Trac website to find out how their dev team advises on these matters. Well, the conclusion is simple: we're using the 'tracd' method of running Trac and this is certainly the only way how *not* to do it.
They advise CGI for small sites or FastCGI/WSGI for larger sites. I estimate us to be in the latter category, so I took my own VPS (which is in the former category, but also runs Trac) and configured it for FastCGI. The steps turn out to be quite simple. I volunteer to make the same changes on c-l.net (which drewc already set me up for), but I'm posting here for review. The last thing I'd want is to open up a security hole in the site.
Steps are:
* Install mod_fcgid (although I used mod_fastcgi, which is older and not recommended by the Trac team; it works though) * Add the script handlers and the script alias to the httpd.conf file within the virtual host section for trac.c-l.net:
AddHandler fastcgi-script .fcgi ScriptAlias / /path/to/apache/cgi-bin/trac.fcgi/
* copy the 'web/' subdir of the installed Trac into the cgi-bin directory * copy in the cgi-bin directory "fcgi_frontend.py" to "trac.fcgi" * add to the trac.fcgi script - before the first 'import' statement:
import os os.environ['PYTHON_EGG_CACHE'] = "/tmp/httpd-egg-cache" os.environ['TRAC_ENV_PARENT_DIR'] = "/path/to/parent/trac/project/dirs"
* create /tmp/httpd-egg-cache with the user and group used to run httpd and '700' access settings * restart apache
Comments?
The effect of the change is this: by default, a single trac.fcgi application will be loaded - just as with our current situation. However, on high demand, additional instances will be spawned - contrary to the current situation, where a single tracd will try to handle all requests. When the load is over, the number of servers is scaled back to normal (1).
Bye,
Erik.
clo-devel mailing list clo-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/clo-devel
clo-devel mailing list clo-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/clo-devel
Hans, Drew,
Thanks for your reactions!
On Mon, Mar 8, 2010 at 6:55 PM, Drew Crampsie drewc@tech.coop wrote:
@Erik : please go ahead, sounds great.
@Hans : We've been running behind squid for 1/2 a year now... it helped at first, but the real issue is with dynamic pages, and tracd does not seem to set proper cache control headers for things that can't/don't change.
Also, tracd does not allow a robots.txt, so FCGI will allow us to cut off yahoo and msnbot, who are the real trouble IMO.
Yea, httpd will allow us to map into the trac-served space; meaning we serve robots.txt making sure "heavy" pages don't get called by robots (time line, roadmap and the report lists). Additionally, we can map a favicon.ico and other often-requested static resources into the space.
Also, trac allows - through its "deploy" command - the extraction of its static resources. Those can be served from c-l.net (the main domain) or they can be mapped into the trac space too. [Just figuredt his one out.]
I'm not sure if the latter matters once the bots have been excluded from requests. It would be interesting to analyse trac traffic after we make these changes.
The changes will have to wait for the weekend though: that's when I expect to have more time.
With respect to authentication: Instead of the documented LocationMatch, I read a Location, which ofcourse didn't match... So, the auth part has been cleared up now too!
Bye,
Erik.
Hi,
Just finished making the change. c-l.net is now running its Trac on mod_fastcgi. The only surprise I met was that c-l.net is running Apache 1.3, not 2.0 as I thought.
On Sun, Mar 7, 2010 at 10:27 PM, Erik Huelsmann ehuels@gmail.com wrote:
Since we seem to have lots of issues with our Trac install (most people find it slow; some find its availability disappointing), I thought I'd look at the Trac website to find out how their dev team advises on these matters. Well, the conclusion is simple: we're using the 'tracd' method of running Trac and this is certainly the only way how *not* to do it.
They advise CGI for small sites or FastCGI/WSGI for larger sites. I estimate us to be in the latter category, so I took my own VPS (which is in the former category, but also runs Trac) and configured it for FastCGI. The steps turn out to be quite simple. I volunteer to make the same changes on c-l.net (which drewc already set me up for), but I'm posting here for review. The last thing I'd want is to open up a security hole in the site.
Steps are:
* Install mod_fcgid (although I used mod_fastcgi, which is older and not recommended by the Trac team; it works though) * Add the script handlers and the script alias to the httpd.conf file within the virtual host section for trac.c-l.net:
AddHandler fastcgi-script .fcgi ScriptAlias / /path/to/apache/cgi-bin/trac.fcgi/
* copy the 'web/' subdir of the installed Trac into the cgi-bin directory * copy in the cgi-bin directory "fcgi_frontend.py" to "trac.fcgi" * add to the trac.fcgi script - before the first 'import' statement:
import os os.environ['PYTHON_EGG_CACHE'] = "/tmp/httpd-egg-cache" os.environ['TRAC_ENV_PARENT_DIR'] = "/path/to/parent/trac/project/dirs"
* create /tmp/httpd-egg-cache with the user and group used to run httpd and '700' access settings
I didn't create directory in /tmp, but in /cache instead.
* restart apache
Note that there's an error in the Trac documentation regarding the configuration of Digest authentication: while the section for configuration of Digest auth lists the Apache directive "AuthUserFile", it should have listed "AuthDigestFile". The directive takes the same parameters, but applies to a different module: the former applies to the Basic auth module.
I had to add mod_digest to the apache configuration too, however that was pre-installed, just like mod_fastcgi, so no additional packages had to be retrieved.
That's about it. Oh, given the current performance drewc and I decided against installing a robots.txt: the performance is good enough for now, it looks.
Bye,
Erik.
-
Drew Crampsie -
Erik Huelsmann -
Hans Hübner