# Robots.txt file Generic for Frontpage.. # Chris Mendla # Updated Dec 5 2002 # MODIfied 10 23 06 to add the chaz folder as a disallow # Modified 2 21 07 to block folders with duplicate content # to prevent duplicate content issues. # Allow all spiders access to HTTPDOCS and HTTPSDOCS # but block them from the webstat directory # NOTE - THere is a seperate Robots.txt in the subdomain folder User-agent: * Disallow: /httpdocs/webstat/ Disallow: webstat/ Disallow: /webstat/ Disallow: /cgi-bin/ Disallow: /_borders/ Disallow: /_derived/ Disallow: /_fpclass/ Disallow: /_overlay/ Disallow: /_private/ Disallow: /_themes/ Disallow: /_vti_bin/ Disallow: /_vti_cnf/ Disallow: /_vti_log/ Disallow: /_vti_map/ Disallow: /_vti_pvt/ Disallow: /_vti_txt/ Disallow: /chaz/ Disallow: /old%20/stuff/ Disallow: /old%20stuff/ Disallow: /admin/ Disallow: /tmp/ Disallow: /Backups/ Disallow: /Copy%20of%20_borders/ Disallow: /Templates/ Disallow: /common/