2. Every Page Included Will Be Indexed

And vice versa, even if you’ve excluded a URL in your sitemap, search engine crawlers may still index the page.

XML sitemaps are merely a recommendation to crawlers. It’s important to keep in mind that your site is sending lots of other signals to crawlers. If you really don’t want a page indexed, we’d definitely recommend a robots.txt disallow.

Probably the most tried and true way to see which pages Google is indeed indexing is to perform a site:search.

You can also perform a search for a specific URL to see if it may or may not be being indexed.

XML Sitemap Best Practices

So we have a good idea of what an XML sitemap is, and we’ve looked at a couple of common sitemap misconceptions. Let’s dive into best practices.

1. Use a Tool to Generate Your Sitemap

The first step to properly utilizing sitemaps is to generate one. Unfortunately, they aren’t just magically created. You have a couple of options to do so:

Yoast SEO

If your website was built on WordPress, this is a no-brainer. Yoast offers a free plugin to help you with on-page optimization, indexing, canonicals and sitemap generation.

If you don’t already have the Yoast SEO plugin installed on your site, here’s what to do:

2. Submit to Google Search Console

Now that you have a sitemap, you may be wondering what to do with it.

The first step is to submit it to Google Search Console. This will help Google crawl and index your website, but as we mentioned, this does not guarantee every page included will be indexed, or that every page excluded will be excluded from the index.

Google requires verification that you indeed own the site. If you have Google Analytics set up, it’ll be done for you automatically. If not, there are a few other options for verifying.

C. Navigate to sitemaps.

D. Insert your sitemap URL and click “Submit.”

And you’re all set! Give Google some time to read the sitemap and check back periodically to see if Google has encountered any errors.

3. Prioritize Highest Quality Pages

When it comes to ranking, it would seem Google not only considers the value of the page in question, but also the overall quality of a website.

Let’s say your website has 500 pages, but only 10 are pages containing fantastic content that’s useful to users. The rest are either old and irrelevant blog posts or “utility” pages (log-ins, shopping carts, places to retrieve lost passwords, etc.).

It’s very possible Google would take this as a signal that the vast majority of your website contains low-quality content, thus hurting your chances of ranking your most important pages well.

So keep this in mind as you decide what pages you want included in your sitemap. As we mentioned above, it’s pretty simple:

Include and index pages you want found through search engines

Exclude and no-index pages you wouldn’t want found through search engines

4. Use Noindex

Speaking of including and excluding pages, it’s really important to be consistent. Including a page on your sitemap yet instructing search engines not to crawl it is not a good idea.

So if you don’t want a page included in your sitemap, leave it off, and make sure it’s not being indexed.

You have a few options when it comes to making sure search engines don’t crawl your site. You can utilize meta robots (instructions in the <head> of a page) or your robots.txt file (a single file containing crawler instructions).

Meta robots are probably a safe bet for an older blog post or a utility page, while a robots.txt disallow would make more sense if you’re looking to minimize your crawl budget.

5. Consider Crawl Budget

Speaking of your crawl budget, it’s important to keep this in mind when considering what pages to include or exclude in your XML sitemap.

Put simply, a crawl budget has to do with what Google refers to as a “crawl rate limit.” In other words, Googlebots can’t just crawl every single page on the web anytime they want. There are limits both to Google’s servers and your site’s servers.

If you have a really large site (we’re talking a few thousand URLs), you’ve got to be much more careful when choosing which pages to include than most websites on the web that have far fewer pages.

And there you have it—everything you need to know about XML sitemaps, how to generate them, submit them and use them to boost your SEO.

Once you’ve followed all these steps, make sure you’re keeping a close eye on your website performance with Monitor Backlinks. It’ll track your keywords for you so you always know which of your pages are crawled and ranking on Google.

The Do’s and Don’ts of Submitting Your Site to Search Engines Submitting your site to search engines is still important for SEO success. Mainly because it offers you benefits you can’t get anywhere else. Benefits like: 1. Making sure search engines index your site, 2. Letting search engines know exactly what’s important on your site, and 3. Using search engine-specific tools...