Wednesday, July 23, 2008

What Googlebot does on your site

Once Googlebot is on your site, it crawls each page in turn. When it finds an internal link, it will remember it and crawl it, either later that visit or on a subsequent trip to your site. Eventually, Google will crawl your whole site.

In the next step I will explain how Google indexes your pages for retrieval during a search query. In the step after that I will explain how each indexed page is actually ranked. However, for now the best analogy I can give you is to imagine that your site is a tree, with the base of the trunk being your home page, your directories the ranches, and your pages the leaves on the end of the branches. Google will crawl up the tree like nutrients from the roots, gifting each part of the tree with its all important PageRank. If your tree is well structured and has good symmetry, the crawl will be even and each branch and leaf will enjoy a proportionate benefit. There is (much) more on this later.

No comments: