# FULL access (Excite) User-agent: ArchitextSpider Disallow: # FULL access (Google) User-agent: Googlebot Disallow: # FULL access (Mirago) User-agent: mirago Disallow: # FULL access (SearchUK) User-agent: MegaSheep Disallow: # FULL access (EuroSeek) User-agent: Freecrawl Disallow: # FULL access (Google Image) User-agent: Googlebot-Image Disallow: # FULL access (Lycos) User-agent: Lycos_Spider_(T-Rex) Disallow: # FULL access (whatUseek) User-agent: winona Disallow: # FULL access (Alta Vista) User-agent: Scooter Disallow: # FULL access (FAST/AllTheWeb) User-agent: FAST-WebCrawler Disallow: # FULL access (WhoWhere?) User-agent: whowhere Disallow: # FULL access (Robot) User-agent: robot Disallow: / # FULL access (Crawl) User-agent: crawl Disallow: / # PARTIAL access (All Spiders) User-agent: * Disallow: /cgi-bin/ Disallow: /test/ Disallow: /awstats/ # FULL CONTENT=INDEX,FOLLOW