I recently setup a new dev server and the CF service kept crashing in the middle of the night. Looking at the traffic it was bots from the other side of the planet and they were clearly not listening to the robots.txt file. Since it's a dev server I don't really want them advertising my site so off to google I went for a solution. I came across this simple way to block user agents. I currently use IIS on my dev server so I followed the instructions and woke up the next morning with a dev server that was still running. Yay! I did have an issue with the user agent switcher not working on Firefox on my mac so I never tested. I figured I'd let the bots do my testing. So far it's JP 1, Bots 0 after one day.