SEO Part II: Bots, Spiders, and a Big Fat Lie

May 22nd, 2009

by John R. Sedivy

spider-on-white1 In SEO Part I, I had discussed the difference between search engine sites and systems. This is an important distinction and if properly understood – stands to save you a great deal of time and effort. In today’s article I will describe the concept of ‘bots and spiders and debunk one of the largest SEO myths out there.

Bots & Spiders
Since prior to starting this blog I had heard much talk about bots and spiders in relation to websites and blogs. But what exactly are these creatures? Peter Kent in his book “Search Engine Optimization For Dummies” describes these entities as software owned by large search-index companies who grab web pages and read the information contained within. Their main purpose is to index this information by use of complex algorithms. How these algorithms work is a closely guarded proprietary secret to prevent users from tricking the system.

The terms robots, bots, or spiders can all be used interchangeably and basically refer to the same concept as described in the preceding paragraph. Once submitted to the major search engine systems, your site will be “crawled” by these bots or spiders and your site indexed within the search engine system crawling your site. Indexing is where your keywords and their strategic placement come into play.

The Biggest SEO Myth
Perhaps the biggest myth in the SEO world is that the largest single source of traffic comes from search engines. Up until starting this blog I had believed this to be true, but quickly found that it was a myth. It simply is not true. In actuality search engine traffic is really below the 50% mark, which is still significant, but not as significant as many self proclaimed “SEO gurus” would have you believe.

So where does traffic come from? According to Peter Kent the following are the largest sources of web traffic:

1. Directly typing web address into browser
2. Clicking from another site which references the destination site

Search engines continue to be significant for a variety of reasons. First, under 50% is still a lot when you consider that there are billions of searches conducted each month. Most people intentionally wishing to visit your site but cannot remember your address will search for you specifically in a search engine. Lastly, most people who research purchases begin doing so on the Internet. So, although it is not the largest source of traffic, increasing SEO is worth your time and effort.

I have found the preceding information to be true based on our traffic analysis of CCBbuzz web traffic. We are increasingly receiving traffic from the search engines but it is dwarfed in comparison to readers directly visiting our site and incoming links from other websites. As with traditional means, there is no substitute for word of mouth. Search engine traffic does bring us new readers, and every extra bit of traffic counts. In addition, search engine traffic helps diversify your overall picture of web traffic sources, which spreads the risk should one source of traffic dry up.

To Be Continued…
In the final part of this series I will identify two useful SEO related tools and the advantages these tools offer in your overall SEO effort.

Although significant, search engines are not the largest source of traffic for your website!

-John R. Sedivy of Cape Cod Branding