Let's take a look at how we can use the Site Analysis tool
to quickly review SEO issues with a site. To avoid embarrassing anyone
else by turning the tool loose on their site, I've decided to instead use the
analysis tool on one of my own sites: www.scottgu.com. This is a site I wrote many years
ago (last update in 2005 I think). If you install the IIS SEO Toolkit you
can point it at my site and duplicate the steps below to drill into the SEO
analysis of it.

Open the Site Analysis Tool

We'll begin by launching the IIS Admin Tool (inetmgr) and
clicking on the root node in the left-pane tree-view of the IIS7 admin tool
(the machine name – in this case "Scottgu-PC"). We'll then
select the "Site Analysis" icon within the Search Engine Optimization
section on the right. Opening the Site Analysis tool at the machine level
like this will allow us to run the analysis tool against any remote server (if
we had instead opened it with a site selected then we would only be able to run
analysis against local sites on the box).

Opening the Site Analysis tool causes the below screen to
display – it lists any previously saved site analysis reports that we have
created in the past. Since this is the first time we’ve opened the tool,
it is an empty list. We’ll click the “New Analysis…” action link on the
right-hand side of the admin tool to create a new analysis report:

Figure 2

Clicking the “New Analysis…” link brings up a dialog like
below, which allows us to name the report as well as configure what site we
want to crawl and how deep we want to examine it.

We’ll name our new report “scottgu.com” and configure it to
start with the http://www.scottgu.com
URL and then crawl up to 10,000 pages within the site (note: if you don’t see a
“Start URL” textbox in the dialog it is because you didn’t select the root
machine node in the left-hand pane of the admin tool and instead opened it at
the site level – cancel out, select the root machine node, and then click the
Site Analysis link).

Figure 3

When we click the “Ok” button in the dialog above the Site
Analysis tool will request the http://www.scottgu.com
URL, examine the returned HTML content, and then crawl the site just like a
search engine would. My site has 407 different URLs on it, and it only
took 13 seconds for the IIS SEO Toolkit to crawl all of them and perform
analysis on the content that was downloaded.

Once it is done it will open a report summary view detailing
what it found. Below you can see that it found 721 violations of various
kinds within my site (ouch):

Figure 4

We can click on any of the items within the
violations summary view to drill into details about them. We’ll look into
a few of them below.

Looking at the “description is missing”
violations

You’ll notice above that I have 137 “The
description is missing” violations. Let’s double click on the rule to
learn more about it and see details about the individual violations.
Double clicking the description rule above will open up a new query tab that
automatically provides a filtered view of just the description violations
(note: you can customize the query if you want – and optionally export it into
Excel if you want to do even richer data analysis):

Figure 5

Double clicking any of the violations in the list above will
open up details about it. Each violation has details about what exactly
the problem is, and recommended action on how to fix it:

Figure 6

Notice above that I forgot to add a
<meta> description element to my photos page (along with all the other
pages too). Because my photos page just displays images right now, a
search engine has no way of knowing what content is on it. A 25 to 150
character long description would be able to explain that this URL is my photo
album of pictures and provide much more context.

The “Word Analysis” tab is often useful when
coming up with description text. This tab shows details about the page
(its title, keywords, etc) and displays a list of all words used in the HTML
within it – as well as how many times they are duplicated. It also allows
you to see all two-word and three-word phrases that are repeated on the
page. It also lists the <a> text used on other page to link to this
page – all of which is useful to come up with a description:

Figure 7

Looking at the URL is linked using
different casing violations

Let's now at the “URL is linked using
different casing” violations. We can do this by going back to our summary
report page and by then clicking on this specific rule violation:

Figure 8

Search engines count the number of pages on
the Internet that link to a URL, and use that number as part of the weighting
algorithm they use to determine the relevancy of the content the URL
exposes. What this means is that if 1000 pages link to a URL that talks
about a topic, search engines will assume the content on that URL has much
higher relevance than a URL with the same topic content that only has 10 people
linking to it.

A lot of people don’t realize that search
engines are case sensitive, though, and treat differently cased URLs as
different actual URLs. That means that a link to /Photos.aspx and
/photos.aspx will often be treated not as one URL by a search engine – but
instead as two different URLs. That means that if half of the incoming
links go to /Photos.aspx and the other half go to /photos.aspx, then search
engines will not credit the photos page as being as relevant as it actually is
(instead it will be half as relevant – since its links are split up amongst the
two). Finding and fixing any place where we use differently cased URLs
within our site is therefore really important.

If we click on the “URL is linked using
different casing” violation above we’ll get a listing of all 104 URLs that are
being used on the site with multiple capitalization casings:

Figure 9

Clicking on any of the URLs will pull up details about that
specific violation and the multiple ways it is being cased on the site.
Notice below how it details both of the URLs it found on the site that differ
simply by capitalization casing. In this case I am linking to this URL using a
querystring parameter named "AlbumId". Elsewhere on the site I
am also linking to the URL using a querystring parameter named "albumid"
(lower-case “a” and “i”). Search engines will as a result treat these
URLs as different, and so I won’t maximize the page ranking for the content:

Figure 10

Knowing there is a problem like this in a site is the first
step. The second step is typically harder: trying to figure out all the
different paths that have to be taken in order for this URL to be used like
this. Often you'll make a fix and assume that fixes everything - only to
discover there was another path through the site that you weren't aware of that
also causes the casing problem. To help with scenarios like this, you can click
the "Actions" dropdown in the top-right of the violations dialog and
select the "View Routes to this Page" link within it.

Figure 11

This will pull up a dialog that displays all of the steps
the crawler took that led to the particular URL in question being executed.
Below it is showing that it found two ways to reach this particular URL:

Figure 12

Being able to get details about the exact
casing problems, as well as analyze the exact steps followed to reach a
particular URL casing, makes it dramatically easier to fix these types of
issues.

Looking at the page contains multiple
canonical format violations

Fixing the casing issues like we did above is
a good first step to improving page counts. We also want to fix scenarios
where the same content can be retrieved using URLs that differ by more than
casing. To do this we’ll return to our summary page and pull up the “page
contains multiple canonical format violations” report:

Figure 13

Drilling into this report lists all of the URLs on our site
that can be accessed in multiple “canonical” ways:

Figure 14

Clicking on any of them will pull up details about the
issue. Notice below how the analysis tool has detected that sometimes we refer
to the home page of the site as "/" and sometimes as
"/Default.aspx". While our web-server will interpret both as
executing the same page, search engines will treat them as two separate URLs -
which means the search relevancy is not as high as it should be (since the
weighting gets split up across two URLs instead of being combined as one).

Figure 15

We can see all of the cases where the /Default.aspx URL is
being used by clicking on the “Links” tab above. This shows all of the
pages that link to the /Default.aspx URL, as well as all URLs that it in turn
links to:

Figure 16

We can switch to see details about where and
how the related “/” URL is being used by clicking the “Related URLs” drop-down
above – this will show all other URLs that resolve to the same content, and
allow us to quickly pull their details up as well:

Figure 17

Like we did with the casing violations, we can use the “View
Routes to this Page” option to figure out of all the paths within the site that
lead to these different URLs and use this to help us hunt down and change them
so that we always use a common consistent URL to link to these pages.

Note: Fixing the casing and canonicalization issues for all
internal links within our site is a good first step. External sites might
also be linking to our URLs, though, and those will be harder to all get
updated. One way to fix our search ranking without requiring the
externals to update their links is to download and install the IIS URL Rewrite module
on our web server (it is available as a free download using the Microsoft Web Platform Installer). We can then
configure a URL Rewrite rule that automatically does a permanent redirect to
the correct canonical URL – which will cause search engines to treat them as
the same (read Carlos’ IIS7 and URL Rewrite: Make your Site SEO blog post to learn
how to do this).

Looking up redirect violations

As a last step let’s look at some redirect violations on the
site:

Figure 18

Drilling into this rule category reminded me of something I
did a few years ago (when i transferred my blog to a different site) - that I
just discovered was apparently pretty dumb.

When I first setup the site I had originally had a simple
blog page at: www.scottgu.com/blog.aspx
After a few weeks, I decided to move my blog to weblogs.asp.net/scottgu. Rather
than go through all my pages and change the link to the new address, I thought
I’d be clever and just update the blog.aspx page to do a server-side redirect
to the new weblogs.asp.net/scottgu
URL.

This works from an end-user perspective, but what I didn’t
realize until I ran the analysis tool today was that search engines are not
able to follow the link. The reason is because my blog.aspx page is doing
a server-side redirect to the weblogs.asp.net/scottgu
URL. But for SEO reasons of its own, the blog software (Community Server)
on weblogs.asp.net is in turn doing a
second redirect to fix the incoming weblogs.asp.net/scottgu
URL to instead be http://weblogs.asp.net/scottgu/
(note the trailing slash is being added).

According to the rule violation in the Site Analysis tool,
search engines will give up when you perform two server redirects in a row. It
detected that my blog.aspx redirect links to an external link that in turn does
another redirect - at which point the search engine crawlers give up:

Figure 19

I was able to confirm this was the problem without having to
open up the server code of the blog.aspx page. All I needed to-do was click the
"Headers" tab within the violation dialog and see the redirect HTTP
response that the blog.aspx page sent back. Notice it doesn't have a trailing
slash (and so causes Community Server to do another redirect when it receives
it):

Figure 20

Fixing this issue is easy. I never would have realized I
actually had an issue, though, without the Site Analysis tool pointing me to
it.