Introduction to Indexing

The indexing of your content by Google is determined by system algorithms that
take into account user demand and quality checks. You can influence the Google
indexing process by how you manage the discovery of your content, which relies
on the URL of the page. Without your pages’ URLs, our systems cannot crawl,
index, and ultimately present your information in Search. This document
introduces the notion of getting into the Google index by helping you decide
how to manage discovery of your content by Google, which is the first step in
the indexing process.

Ways to manage your resources for discovery by Google

You can choose a number of approaches to help Google find your resources and
data, from entirely passive to very pro-active. This section describes the
general choices you might want to make to providing resource metadata—sitemaps
and resource linking—to best position your content to appear in Search.

A. Take a passive approach

If you create a website without providing a sitemap, our systems attempt to
find and index the content on your site unless you specifically block your
content from crawlers. As a normal course of business, Google systems crawl
the relationships between your pages and other website pages that link to your
content. For more information, see the
Introductory Guide.

Advantages: No additional work beyond producing your content. This
approach is adequate for situations where you have a simple website and do not require timely
content discovery in Search results.

Disadvantages: Relying on natural links as a sole means to discovery means
that our systems might not be able to find all the content on your site, especially if it is
new or has few references. This might cause problems for new content that you are eager to see
in Search, such as content you mark up especially for inclusion in rich results (previously known as rich cards).

B. Actively manage your URLs

By providing our systems a direct list of URLs to your content, known as a
sitemap, our ability to find your pages no longer relies solely on your page’s
relationship to other referring pages on the wider web. This speeds up the
process of our systems discovering your content. Typically, you host the
sitemap on your domain in a place accessible by Googlebot.

In addition, if you have multiple URLs that contain essentially the same
content—such as an AMP page, and HTML page, and a mobile app view—it helps to
indicate that relationship between those resources. Establishing the
relationship between your resources allows our system to correctly serve the
right content, such as a link to your app or to your AMP pages. To do this,
you establish the canonical pages for your site and set up a linking
relationship between those pages and alternative web or app content. Once we
establish the relationships between your various resources, we can determine
which type of content to display to the user in Search results, such as showing
a link to your app for users who search from their phone and have your app
already installed.

Advantages: Enhances the performance of your rich results in Search. Speeds up the process
of ingesting new and low-referral content into the system. This approach
removes one potential obstacle for getting your content quickly served by
Google in a variety of forms.

Disadvantages: You do the additional work of providing resource
metadata, which is your sitemap and relationships you specify between your web pages, your
app, and your AMP pages.

C. Submit new and updated URLs to Google

While you can merely host your sitemap on your site for our systems to
discover, you can also provide notification about new URLs or existing URLs
that have changed content.

For new URLs, submitting your sitemap helps us more quickly discover them. For
content changes to existing URLs, you can provide an XML sitemap with
modification timestamps to notify us of changed content ready for re-indexing.

When our systems receive the list of URLs, we then determine when to crawl the
content. For the content that we do crawl, we establish the existence of the
resource on your server—a process known as verifying—and then make that content
ready for our indexing process.

D. Get your app indexed

Firebase App Indexing, formerly Google App Indexing, gets your app into Google Search. If
users have your app already installed when they search for related content, your app launches
directly from the Search results. If users don’t yet have your app, they will see an install
card in Search results when they search for apps. The App Indexing API promotes both types
of app queries within Search, and also provides query autocompletions.
Read more about Firebase App Indexing.

Advantages: Using App Indexing for your app adds a positive
ranking signal for all your content, app and website.

Disadvantages: Additional content management for your app
and website. The work, however, is greatly simplified by using Android Studio as
your development environment, which streamlines the resource management. Learn
more about
using Android Studio for App Indexing.