Home > Biz, Blog, Tech, Web > Blocked From Spidering

Blocked From Spidering

April 13th, 2010

It seems that some websites are blocking search engines from spidering some parts of their sites. This includes website like Craiglist and Del.icio.us.

Last year, for example, many SEO websites were reporting that Yahoo has blocked other search engines from accessing certain links on Del.icio.us. What did it mean to an average Internet user? I do not think it really had such an impact, although it might have been a problem for webmasters who were receiving many backlinks from Del.icio.us in hopes to improve their site rankings. Those of us who had many incoming links from Del.icio.us might have noticed less links being indexed by Google. Since PageRank is determined by the number and quality of links, having less links might have even resulted in Page Rank dropping on some of the websites.

This situation only proved that our marketing and promotion strategy should be diversified. It should include many different websites from which we hope to receive links from. If you advertise using many different methods, then you reduce the chance of anything going wrong. After all, there is very small probability that all of your marketing methods will stop working at the same time.

Categories: Biz, Blog, Tech, Web Tags:
Comments are closed.