Watching the spiders crawl
One of the interesting things about relaunching the debabblog is that it's the first time I've really gotten to see the search engine spiders "discover" a site. Most of the sites I've worked on have been around for a few years, long before tools like Google Webmaster Tools and Yahoo! Site Explorer came into being. Watching a new site take root in the search indexes has given me a new perspective.For starters, it became immediately clear, and a Google rep all but confirmed this recently, that Google is a bit more stingy with information than Yahoo! is. Most notably, Google's link tracking (type "link:www.yourdomain.com" into the Google search to see what pages link back to you) only shows a relatively small sample of linked sites. In my case, Google registers a big fat zero links, whereas Yahoo! has over fifty. Even with the new site map protocol up and running, Google also has only indexed my home-page, whereas Yahoo! has crawled about a dozen pages.
In the beginning, the spiders were checking my site less than once/week. This makes a certain sense, as "debabblog.com" previously redirected visitors to my company website, and that blog was only updated on a weekly basis. Now that I'm producing content more frequently, the spiders seem to be getting a little friendlier.
Mostly, it makes me realize that building links "organically" is just like starting any new venture. The beginning is the hardest part, because you have no contacts and no credibility. When things start to happen, though, they have a way of taking off. Suddenly, all of the links that you've built up get noticed, your word-of-mouth kicks in, and each new connection gets that much easier to build. In that way, search engines are already beginning to mimic the way that social trends really work.
