User-Agent: localhtdig Disallow: User-Agent: *msnbot* Disallow: / User-Agent: * Disallow: /cgi-bin/ Disallow: /usage/ Disallow: /perl/ace/ Disallow: /perl/ace/pic/ Disallow: /perl/ace/tree/ Disallow: /perl/ace/search/ Disallow: /perl/ace/alignm/ Disallow: /perl/ace/goBlast/ Disallow: /perl/ace/dnaDump/ #User-Agent: *gsa-crawler* #Disallow: /