user-agent: AhrefsBot User-Agent: Slurp User-Agent: Teoma User-Agent: Scooter User-Agent: ia_archiver User-Agent: Lycos User-Agent: WebAlta User-Agent: MJ12bot User-agent: grapeshot User-Agent: Aport User-Agent: Baiduspider User-Agent: trendictionbot User-Agent: dotbot Disallow: / User-agent: bingbot User-Agent: Mail.Ru User-agent: msnbot User-agent: SputnikBot Crawl-delay: 2 Disallow: /cat/descsc.html? Disallow: /cat/product-info.html? Disallow: /*.pdf Disallow: /?order= Disallow: /?preview= # old bad links Disallow: /?search= Disallow: /?p=search Disallow: /?p=question Disallow: /p/questions/?p=question Disallow: /search/?q= Disallow: /faq Disallow: /html/ Disallow: /?p=nku User-agent: * Disallow: /cat/descsc.html? Disallow: /cat/product-info.html? Disallow: /*.pdf Disallow: /?order= Disallow: /?preview= # old bad links Disallow: /?search= Disallow: /?p=search Disallow: /?p=question Disallow: /p/questions/?p=question Disallow: /search/?q= Disallow: /faq Disallow: /html/ Disallow: /?p=nku Host: www.etm.ru