Wednesday, July 23, 2008

How Googlebot first finds your site

There are essentially four ways in which Googlebot finds your new site. The first and most obvious way is for you to submit your URL to Google for crawling, via the “Add URL” form at www.google.com/addurl.html. The second way is when Google finds a link to your site from another site that it has already indexed and subsequently sends its spider to follow the link. The third way is when you sign up for Google Webmaster Tools (more on this on page 228), verify your site, and submit a sitemap. The fourth (and final) way is when you redirect an already indexed web-page to the new page (for example using a 301 redirect, about which there is more later).

In the past you could use search engine submission software, but Google now prevents this – and prevents spammers bombarding it with new sites – by using a CAPTCHA, a challenge-response test to determine whether the user is human, on its Add URL page. CAPTCHA stands for Completely Automated Public Turing test to tell Computers
and Humans Apart, and typically takes the form of a distorted image of letters and/or numbers that you have to type in as part of the submission.

No comments: