Does Your Website Have a Site Map? It Should!

• HTML site maps. There's the site map that's on your website that people can see (like the sitemap on the Webstix.com website). It contains links to every page on your site. This is done for two main reasons.
  
  First, for robots/spiders (programs sent out by search engines to "crawl" your website and gather information about what is on it so that this information can be put into search results). The HTML site map makes sure they know where every page is.
  
  Second, HTML site maps can help people find a page on your web site that they know is there. People use websites in different ways. Some will use the navigation, some will use the site search, others will go for the site map and some will use all 3 ways.
  
• XML site maps. This site map is an XML formatted document that's not really seen by people visiting your website. It's in a format that the search engine spiders really like and it's more efficient for them to just use the XML site map. They look like this (snippet):
  
  <url>
<loc>http://www.webstix.com/about_us.html</loc>
<priority>0.40</priority>
<changefreq>daily</changefreq>
<lastmod>2009-01-08T00:05:07+00:00</lastmod>
</url>

Having an XML site map on your website can help speed up the amount of time that spiders spend on your site:

Report: Sitemaps Decrease Crawler Response Time

An SEOMoz post has documented statistics on how submitting a Sitemap to Google and Yahoo impacts the rate of crawling that web site. The results show that submitting a Sitemap to Google and Yahoo decreased the time it took Google and Yahoo to crawl the page.

...

It is true that many sites see their content indexed within minutes, with or without a sitemap file. But in some cases, it takes longer.

Do Sitemaps Affect Crawlers?

The experiment was to see if submitting a Sitemap to Google and Yahoo would decrease the time it took Google to crawl and index the page. The results for this blog were amazing! When a Sitemap was submitted the average time it took for the bot to visit the new post was 14 minutes for Google and 245 minutes for Yahoo. When no Sitemap was submitted and the bot had to crawl to the post, it took 1375 minutes for Google and 1773 for Yahoo.

Why is this important? Well, as the second article states, it's just faster. You get into Google or Yahoo's database faster. That's good. Basically, it's to your advantage to be a good host for the search engines that visit your website.

We are including an automatic XML site map generator with our websites. It runs either once a week or nightly and then automatically updates your site map and it also submits that sitemap to Google so that you're always up to date. We've been doing this a while and we're seeing great results. Some websites get indexed very quickly and we see new pages show up on Google the very next day.

This is just another reason to blog on your website. You never know when a certain post will attract a lot of attention. If your website's set up to work well (site maps, a CMS, good SEO, etc.) you'll get a boost of traffic. We've seen that with a few of my blog posts already. It's pretty cool and fun when that happens - you just have get blogging.

-Tony

Comments (0)

Subscribe to this comment's feed