As you may be aware one ranking factor that Google uses to work out what sites to favour is the web site loading speed.
Slow or unreliable sites tend to get penalised in the SERPS. If Google has to wait too long to spider and index your web pages, there is a chance those pages will not get top results.
The trouble is there are now hundreds of computers on the internet proding, poking and scraping your web site looking for vulnerabilities, trying to submit spam comments, profiling your site and stealing your content.
These “bots” as they are called, slow your web site to a crawl. The majority of these bots reside in China, France, the Netherlands and Eastern Europe.
You may not notice that your web site is slow because you may be loading a browser-cached copy of your web pages.
If you want to see a fascinating real-time graphic of this world-wide bot traffic, click here: http://tinyurl.com/nv2693s Note that this page only show’s the top 2% of attacks.
If there are enough of them, bots can crash your site and force your hosting provider to suspend your hosting account. Not a good thing. And not good for your SEO.
So what can you do?
If you know what you’re doing you can block bots using a number of criteria, these include their IP address, browser user agent and visitor behaviour.
If your site is run on WordPress there are a bunch of plugins that you may find useful.
Here’s a few to search for:
AP HoneyPot WordPress Plugin
Project Honey Pot Spam Trap
iQ Block Country
IP2Location Country Blocker
Read the documentation carefully to see if any of these plugins are appropriate to you. DO NOT INSTALL THEM ALL.
For those of you who don’t use WordPress, I suggest you read this blog post about modifying your server’s .htaccess file http://code.tutsplus.com/articles/secure-your-wordpress-against-user-agents-and-bots–wp-24345
