User-agnet: Gigabot Disallow: / User-agent: NaverBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: FAST-WebCrawler Disallow: / User-agent: Mercator Disallow: / User-agent: NPBot Disallow: / User-agent: ZyBorg Disallow: / User-agent: Slurp Disallow: / User-agent: RoboCrawl Disallow: / User-agent: OmniExplorer Disallow: / # Why is your bot excluded? # You're reselling, commercially, information taken from my site without my # permission in violation of the license I've made the information available under. # You're wasting bandwidth, harvesting billions of web pages for some pisswater # search engine that no one uses, and not even making the harvested pages available # to other pisswater searchengines so that they don't have to. # In short, you're a drain on humanity, a mooch, a leech, a user, and I don't # want to help you make a quick buck at society's expense. # Slurp makes more passes through the site than all the other search engines # put together. # Microsoft fudges it's search results to smear Linux - what on Earth # could they want with these pages then? Are you crawling these pages # so you know what keywords to send to pages about C#? Did you ever stop # to think that maybe Windows users port applications from Linux and # need documentation just as much as anyone else? Anyway, I don't like # how you play the game. Go away. I ain't gunna tango wit'chews. User-agent: msnbot Disallow: / # Why are these excluded? # They are reports that are extremely CPU intensive to generate, but contain # no information that isn't already presented in the normal view, or else # they are test versions of scripts that present the exact same information # but have a different URL. Thank you for your understanding. User-agent: GoogleBot Disallow: /index.cgi Disallow: //reverse.cgi Disallow: //diff.cgi Disallow: //metric.cgi Disallow: //assemble.cgi Disallow: //rate.cgi Disallow: // Disallow: /// Disallow: //// Disallow: ///// Disallow: ////// # GoogleBot, you have issues Disallow: /metric.cgi Disallow: /reverse.cgi Disallow: /assemble.cgi Disallow: /spell.cgi Disallow: /reverse.cgi Disallow: /diff.cgi Disallow: /rate.cgi Disallow: /wiki.cgi Disallow: /wiki2.cgi Disallow: /wiki3.cgi Disallow: /tinywiki1.cgi Disallow: /tinywiki2.cgi Disallow: /tinywiki3.cgi Disallow: /tinywiki4.cgi Disallow: /wikia.cgi Disallow: /wikib.cgi Disallow: /wikic.cgi Disallow: /wikid.cgi Dissalow: /intermap.cgi Disallow: /WikiWikiList Disallow: /?WikiWikiList Disallow: //?WikiWikiList User-agent: ia_archiver Disallow: /index.cgi Disallow: //reverse.cgi Disallow: //diff.cgi Disallow: //metric.cgi Disallow: //assemble.cgi Disallow: //rate.cgi Disallow: // Disallow: /// Disallow: //// Disallow: ///// Disallow: ////// Disallow: /metric.cgi Disallow: /reverse.cgi Disallow: /assemble.cgi Disallow: /spell.cgi Disallow: /reverse.cgi Disallow: /diff.cgi Disallow: /rate.cgi Disallow: /wiki.cgi Disallow: /wiki2.cgi Disallow: /wiki3.cgi Disallow: /tinywiki1.cgi Disallow: /tinywiki2.cgi Disallow: /tinywiki3.cgi Disallow: /tinywiki4.cgi Disallow: /wikia.cgi Disallow: /wikib.cgi Disallow: /wikic.cgi Disallow: /wikid.cgi Dissalow: /intermap.cgi Disallow: /WikiWikiList Disallow: /?WikiWikiList Disallow: //?WikiWikiList # You know what? People create these faster than I can shitlist them, so # you dinks have ruined it for everyone. Unless you're Google, go away. User-agent: * Disallow: /index.cgi Disallow: /WikiWikiList Disallow: /?WikiWikiList Disallow: //?WikiWikiList Disallow: /metric.cgi Disallow: /reverse.cgi Disallow: /assemble.cgi Disallow: /spell.cgi Disallow: /reverse.cgi Disallow: /diff.cgi Disallow: /rate.cgi Disallow: /wiki.cgi Disallow: /wiki2.cgi Disallow: /wiki3.cgi Disallow: /tinywiki1.cgi Disallow: /tinywiki2.cgi Disallow: /tinywiki3.cgi Disallow: /tinywiki4.cgi Disallow: /wikia.cgi Disallow: /wikib.cgi Disallow: /wikic.cgi Disallow: /wikid.cgi Disallow: /