User-agent: Baiduspider Allow: / User-agent: Googlebot Allow: / User-agent: 360Spider Allow: / User-agent: Yisouspider Allow: / User-agent: Bytespider Allow: / User-agent: dotbot Disallow: / User-agent: Abonti Disallow: / User-agent: ADmantX Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BacklinkCrawler Disallow: / User-agent: BPImageWalker Disallow: / User-agent: check_http Disallow: / User-agent: checks.panopta.com Disallow: / User-agent: curl Disallow: / User-agent: DLE_Spider.exe Disallow: / User-agent: dotbot Disallow: / User-agent: EMail Exractor Disallow: / User-agent: Exabot Disallow: / User-agent: Ezooms Disallow: / User-agent: GbPlugin Disallow: / User-agent: gooblog Disallow: / User-agent: gsa-crawler Disallow: / User-agent: HolmesBot Disallow: / User-agent: ichiro Disallow: / User-agent: Infoseek SideWinder Disallow: / User-agent: Infoseek SideWinder/2.0B (Linux 2.4 i686) Disallow: / User-agent: ip-web-crawler.com Disallow: / User-agent: Jakarta Commons-HttpClient Disallow: / User-agent: Java Disallow: / User-agent: JikeSpider Disallow: / User-agent: libwww-perl Disallow: / User-agent: linkdex.com Disallow: / User-agent: Mail.RU_Bot Disallow: / User-agent: MJ12bot Disallow: / User-agent: MLBot Disallow: / User-agent: moget Disallow: / User-agent: NaverBot Disallow: / User-agent: PEAR HTTP_Request class Disallow: / User-agent: PHP Disallow: / User-agent: Pixray-Seeker Disallow: / User-agent: PycURL Disallow: / User-agent: python-requests Disallow: / User-agent: Python-urllib Disallow: / User-agent: rmnbot Disallow: / User-agent: Ruby Disallow: / User-agent: Screaming Frog SEO Spider Disallow: / User-agent: SemrushBot Disallow: / User-agent: SEOENGWorldBot Disallow: / User-agent: SeznamBot Disallow: / User-agent: sitebot Disallow: / User-agent: Squider Disallow: / User-agent: ssearch_bot Disallow: / User-agent: suchmaschinenoptimierung.de Disallow: / User-agent: Synapse Disallow: / User-agent: Wget Disallow: / User-agent: woopingbot Disallow: / User-agent: Wotbox Disallow: / User-agent: Xenu Link Sleuth Disallow: / User-agent: Yandex Disallow: / User-agent: Yeti Disallow: / User-agent: seoprofiler Disallow: / User-agent: LieBaoFast Disallow: / User-agent: PetalBot Disallow: / User-agent: adsbot Disallow: / User-agent: * Crawl-delay: 30