Reader's Favorite Articles
Recent Hits All Time Matt's Favorites

Written Lessons

You are viewing a single article.

Lesson 2 Vocab 20 - Crawl and Indexing

Earlier I mentioned the term "bot" and how it refers to the Google Spider that reads websites. This Google Spider has the official name of "googlebot" which you will frequently see in your analytics reports.

Many newbies to the internet get confused between the words "Spider" and "Crawl" and "Indexing" because they all relate to a similar action. But there are subtle differences that will help your completely understand the SEO process.

First, as you know there is a Google program, referred to as a "Spider," or "googlebot." This is the software used to make a copy of your website.

Second, the process of making a copy of your website is called "crawling." You may also hear the term "spidering" but in 2009 Google made a shift in several of its helpful tools and replaced the words "spider" and "spidering" with the word "crawl." Ever since then I have witnessed more people adopting the new "crawl" terminology. Again, crawling is the process of downloading and making a copy of your website for later analysis.

Third, "indexing" is the process whereby the Google servers read through your downloaded website and strips out all your content. All the search engines have the ability to ignore your graphics, navigation, buttons and all the code that makes your website work. They simply look for the largest cluster of content.

Referring back to the discussion of "content," I said you need to have at least 250 words on your page. If you have less than 250 words the search engines might not realize you even have content because your words might accidentally get ignored just like they ignore the navigation, images and logos.

Google provides an online service called Webmaster Tools. This service allows you to see vital statistics about your website. One statistic is the last time your website was crawled.

Please don't get too excited when you realize that Google crawls your website every day. The crawl rate doesn't really matter. What matters is how often your website goes through the indexing process.

You will not show up in the search engine results until your website has been properly indexed. One common question you will hear when talking to a SEO professional is "When was the last time your site was indexed?" They always ask this question as a way to show off their prowess so they can then demonstrate that they know how to look up your latest indexing date.

Here's an exercise for you.
      1. Go to
      2. Type in
      click [Google Search]
      Your website will show up on the search results page
      3. Click the Cashed link next to your green website address
      4. View the date shown next to "as it appeared on..."
      That date is the last time your website was "indexed."
      5. You are now as clever as a professional SEO.
Confused and worried about your mobile website options? Click here to find out how to get your own website evaluation and a game plan to make it better.

Like This Article? Please Share!

Like Our Site? Follow Us!

AT: 01/22/2010 02:13:03 PM   LINK TO THIS ARTICLE

Post a Comment

Check here for Anonymous


Please contact me at the phone number and address below
Phone Number



User Verification
8 6 7 8 4 7 1 5
Please enter the number you see in the box.
[ What's This? ]
Sign Up For Emailed Daily Gold Nuggets
"...serious kudos to you. We love your straight talk, pertinent information and plain language. I don't know how many industries have something of jWAG's caliber available, but I learn from the emails every day. Really, really nice work, and very appreciated."
-Cheryl Herrick, Global Pathways Jewelry