TWIMC

95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET /robots.txt HTTP/1.1” 200 – “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET /robots.txt HTTP/1.1” 200 – “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:06 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:07 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”
95.108.150.235 – – [03/Feb/2011:18:41:07 +0300] “GET / HTTP/1.1” 200 806 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)”

Мне кажется, или бот зря берет одну и ту же страницу несколько раз за 1 секунду? Или он так убеждается, что www.andreybondarenko.com и andreybodarenko.com – зеркала?


Posted

in

,

by

Tags:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *