95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET /robots.txt HTTP/1.1” 200 – “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET /robots.txt HTTP/1.1” 200 – “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:07 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:07 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
Мне кажется, или бот зря берет одну и ту же страницу несколько раз за 1 секунду? Или он так убеждается, что www.andreybondarenko.com и andreybodarenko.com – зеркала?
Leave a Reply