User-agent: * crawl-delay: 1 Disallow: /*/keres.php?*offset=