# 2007.05.18.rh Fine tuning # 2007.03.21.rh Robots.txt # # # Let them loose on the home page. # Keep them out of everywhere else. # # # ttd: - Be more specific about which pages to index. # - Add meta-tags to the documents. Just remember many robots # don't use these tags # # If disallow parm ends with a /, ignore everything in that directory # else ignore everything that matches (including directory) # #User-agent: NetMechanic #Allow: / # User-agent: * Allow: / Allow: /RAAP Article.pdf Allow: /RAAP Presentation.ppt Allow: /ClientProcess.png Allow: /HostProcess.png #Disallow: /index.php Disallow: /CSM* Disallow: /TOUR* Disallow: /VE* User-agent: msnbot Disallow: / # User-agent: IconSurf Disallow: / #