How Do Search Engines Do That?

Searching for Websites and information online is a straightforward task. Simply type a phrase into a search engine and you're served with lists of results from all over the Web. But how to search engines find, understand, catalog and serve up those search results? Search engines do all of this in three steps - crawling, indexing and serving results. Let's take a look at the process that search engines go through to provide Internet users with quality search results.

Crawling

How do search engines know about your site? How do they find you online?

Search engines use spiders (also known as robots) to find and read Websites. These spiders 'crawl' billions of Web pages, looking for new and updated content. Search engines typically use these crawlers to find and create a copy of all the pages they visit. During the crawling process, search engines follow the links within a Website. From page to page and from other sites, links are key throughout the crawling process. Having search engine friendly href links is vital to search engine rankings because of how search engine spiders crawl the Web.

Indexing

Do search engines understand your Website? How do they comprehend what your Website is about?

Once a search engine's robot or spider finds your Website, they need to process and store the information you have presented. Once the search engines crawl your Website for the indexable information, they store that information for future use. Search engines put together a giant resource, an index, of the content and subject matters of crawled Websites.

Search engine crawlers "read" text, alt tags, images and links, and process that information while indexing a site. A search engine can crawl your Website, but that doesn't mean that the content has been indexed. In fact, not all content can be indexed by search engines. Providing these crawlers with information they can process and store is an integral part of receiving high search visibility. Once your Website has been indexed, you have the opportunity to be listed on search engine results pages. Just getting indexed does not guarantee you will be displayed in top search positions.

Serving Results

How do search engines decide on the order of search results?

Search engines access the stored information from indexing billions of Websites, looking for the most relevant results. Search engines use complicated algorithms to determine which Websites appear where on search engine results pages. Google, for example, uses over 200 factors to determine the relevancy of a Website, and where to place it in search engine results. Some of the most influential factors search engines take into account include the quality of links to and from the Website, where and how important keywords are used on the page, and the popularity of those pages with searchers. Search engine result pages are not fixed. As Websites change, search engines reorder their results. This flexibility allows Websites to adjust and modify their sites in order to become more relevant and targeted, resulting in better search positions.

5 tips for making your Website Search Engine Friendly:

Make sure your content is indexable, not hidden behind Flash, JavaScript or other showy technology.

Utilize important Meta Data tags. Page titles and Meta Descriptions are important for search engines to determine your site's topic and relevancy to users' searches.

Create a robots.txt file and Site Map to help crawlers decipher your site.

Set up and utilize Google Webmaster tools. You will see when your site was last crawled and if there are any errors hindering the search engines.

Maintain your Website. Crawlers make repeat visits to Websites, probably every day. Update your site with new, relevant content to show search engines that you have an active Website worthy of search positions.

The Bottom Line

Search engines need to be able to crawl and index your Website before they can deem it relevant and serve it to searchers. You cannot receive high search visibility without taking into account the processes search engines use to present Internet users with search results. There are many important optimization steps to take to assure your Website can and will be understood and indexed by search engines. Understanding the process is one of the first steps in gaining search visibility for your Website. You may have a great looking Website with all the bells and whistles, but if it is not search engine friendly, neither the search engines nor your visitors will be able to find it.

Leave a comment

As Director, SEO and Social Media at E-Power Marketing, Adrian is primarily focused on the organic success of client programs through SEO, social media and content marketing. Adrian also heads up the agency's marketing initiatives. With a background in psychology and PR, her passion for telling a brand's story by understanding and engaging with their target audiences has been put to good use at E-Power!