Created a robots.txt for viewvc and gitweb. System load now down below 10. If/when required, I'll look at creating a robots.txt for Trac as well. That's more involved due to the lack of using URL patterns in robots.txt.
Regards,
Erik.
On Tue, Jun 17, 2014 at 10:57 AM, Erik Huelsmann ehuels@gmail.com wrote:
Hi,
Seems like cl-net is being crawled by bots. there are lots of active trac processes as well as viewcgi processes. Which seems logical: there used to be a pretty extensive robots.txt file which should prevent scanning gitweb, viewcgi and large parts of trac. apparently, that's not there anymore.
Regards,
Erik.
On Tue, Jun 17, 2014 at 6:24 AM, Raymond Toy toy.raymond@gmail.com wrote:
On 6/14/14 2:11 PM, Anton Vodonosov wrote:
For example: http://trac.common-lisp.net/cmucl
It was up a day or so ago, but it is down again.
Also, as I type this, the load avg on c-l.net is 46, with about 20 viewvc.cgi processes running.
Ray
Clo-devel mailing list Clo-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/clo-devel
-- Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.