On Fri, 14 Dec 2007 17:18:38 -0500, Gary King gwking@metabang.com wrote:
IIRC, there is some way to ping Google and get them to spider your page more quickly...
The text is /not/ a result of spidering - it never appeared on my webpage in this form. Google spiders weitz.de pretty often anyway. For example, Googlebot looks at the Hunchentoot page about once per day:
edi@nanook:/var/log/apache$ nslookup 66.249.67.206 Server: 81.169.163.104 Address: 81.169.163.104#53
Non-authoritative answer: 206.67.249.66.in-addr.arpa name = crawl-66-249-67-206.googlebot.com.
Authoritative answers can be found from:
edi@nanook:/var/log/apache$ cat weitz_access.log.1 weitz_access.log | grep -i 66.249.67.206 | grep "GET /hunchentoot/ " 66.249.67.206 - - [02/Dec/2007:12:10:00 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [03/Dec/2007:12:05:52 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [03/Dec/2007:16:40:12 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [04/Dec/2007:18:51:13 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [05/Dec/2007:16:16:03 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "DoCoMo/1.0/N505i/c20/TB/W20H10 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [05/Dec/2007:22:05:52 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [05/Dec/2007:22:30:16 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [07/Dec/2007:05:49:10 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [07/Dec/2007:08:09:28 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [08/Dec/2007:08:41:29 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [09/Dec/2007:17:13:07 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [11/Dec/2007:14:32:29 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [12/Dec/2007:15:36:36 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [13/Dec/2007:16:05:52 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [14/Dec/2007:15:32:27 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "DoCoMo/1.0/N505i/c20/TB/W20H10 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)" 66.249.67.206 - - [14/Dec/2007:19:47:44 +0100] "GET /hunchentoot/ HTTP/1.1" 200 133631 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"