# ============================================================ # robots.txt - stop webcrawlers/spiders from # filling up log files with irrelevant searches # # Updated 02 April 2013 # ============================================================ User-agent: * Disallow: /history/courses/ # NO LONGER USED BY HISTORY Disallow: /navigation/ # DOES NOT EXIST Disallow: /local/ # internal service tools Disallow: /staff/ # staff/ superceded by staffnet/ Disallow: /hr/ # hr/ superceded by staffnet/ Disallow: /ppg/ # ppg/ superceded by staffnet/ Disallow: /old-hr/ # in case anyone links to the archive Disallow: /old-ppg/ # in case anyone links to the archive Disallow: /modernthought/wiki/ # broken, remov