To make sure that Google’s search engine is returning the right answers, Google regularly uses human contractors to evaluate search results to make sure that the latest algorithm is returning the kind of results they want.Google’s Quality Evaluator Guidelinesare the instructions that they give these evaluators.

The guidelines that Google gives Quality Evaluators give us insight into what Google is aiming for. Even if Quality Evaluators don’t visit the site, meeting these criteria will help organisations match what Google believes is a good quality website. Jennifer Slegg’s piece does a good job of detailing the2019 update to the Search Quality Evaluator Guidelines.

We’ve created a checklist that distills Google’s Quality Evaluator Guidelines and aims to give us more information to succeed in search.

This post and accompanying checklist will allow you to:

Gain insightinto what a website and organisation is doing well and could be doing better, in terms of facilitating a good user experience and the creation of quality content.

Provide a platform for a website to perform well in search, by aligning with Google’s perceived quality attributes.

Get internal buy in. The checklist is consistent, independent, quantifiable and based on Google’s guidelines. It could be filled out by internal teams or used as part of user testing.

Make sure content is optimised for launch. The checklist can be used for internal stakeholders to go through before publishing new content, website features or most things that could impact the UX of a website and content quality.

We then took the questions that Google is trying to answer in each topic and turned them into Pass/Fail items in our survey.

Perhaps not surprisingly, the principle of what Google believes constitutes a high quality website hasn’t changed too much throughout the years, although the qualifying criteria have certainly become more robust.

Weighting of questions

We decided how important a question should be based on wording in Google’s documentation.

For example, Google rates pages “lowest quality” if there is intent to deceive users or search engines.

“We will consider a page to be “deceptive” if it may deceive users or trick search engines. All deceptive pages should be rated Lowest”

Google then goes on to describe misleading titles as a form of deception.

Section 7.6.1Deceptive Page Purpose:

“A webpage with a misleading title or a title that has nothing to do with the content on the page. Users who come to the page expecting content related to the title will feel tricked or deceived”.

As a result of this wording in Google’s documentation, where they deem deception to be indicative of “lowest quality” pages, we have assigned strong relative importance to questions that pertain to deception.

If your site fails the question below, it will fail that entire portion of the checklist, because of the importance that Google has placed on accurate titles.

Does the title in Google Search accurately describe the topic of the page?

Pagetype questions

When you are filling out the checklist, there are some questions which should be asked about multiple different types of pages. The page types include:

Homepage

Product pages

Blog pages

Applying topics to different page types will give a broader view of the overall UX and content quality and help pinpoint strengths and weaknesses.

Of course, some page types may not be relevant to an organisation. For instance, publications don’t have product pages, so asking questions about a product page is not applicable.

Sitewide questions

Over the last year or so Google has really focussed on the profile of organisations and content authors in their search algorithms, with August’s 2018 medic updatebeing of particular note.

The E-A-T (Expertise, Trustworthiness and Authoritativeness) of organisations and content creators is seen by Google to be indicative of quality and value. Importantly, Google places more weight on the external evidence of E-A-T, rather than an organisations’ or content creators’ self-endorsement.

We’ve created specific questions that focus on identifying external reputation, with the idea that external reputation represents the potential E-A-T of a website.

We also created questions that focussed on more hygiene aspects of a website, specifically the adequacy of information on a business/content creators and how the design of a website affects the overall UX.

There could be many actions that could be taken from the checklist. Some insights we’ve provided for clients when running our checklist include:

Customer concerns/complaints on third party websites

External reputation information of a brand is important for both Google and users.

So by identifying negative sentiment towards a brand and/or product on an influential industry related website, we convinced our client to not only review concerns on said website, but commission an off site review investigation for many other related websites.

Content creators lacking perceived expertise

Google says that the E-A-T of content creators is indicative of content quality. Our checklist highlighted that content creators, despite being experienced within the vertical they work in, didn’t have an online presence.

The outcome of this insight was to plan guest writing opportunities on industry publications to enhance employees perceived expertise.

Lack of information on content creators

Users want to learn more about and validate the legitimacy of content creators. Our checklist highlighted that individual content creators didn’t have bio pages.

The outcomes of this insight was to develop individual bio pages, with information on their experience and expertise.

Product page insights

Some common insights include:

Not fully understanding the purpose and/or benefits of a product: often a common issue on B2B websites, this can be the result of jargon heavy content, long-winded descriptions that don’t make the benefits clear, or a lack of FAQ’s.

Page design: content is sometimes difficult to digest. This can be the result of bunched up blocks of text, blurry or broken images, or key content placed near the bottom of the page.

Blog page insights

If executed properly, an organisation’s blog is key in driving top of the funnel traffic. This checklist highlights areas such as:

Inaccurate headlines: users need to have an accurate understanding and expectation of the benefits/knowledge they could gain by reading a blog post. We certainly don’t want to mislead readers in any form.

Fact validation: to enhance the authority of your content, you should link to reputable third party sites to validate facts when possible and appropriate.

Unique content: create more in depth and potentially data driven pieces of content to encourage bookmarking/sharing.

Is the main content on the [PAGE TYPE] created with substantial effort, time and talent/skill?

“We will consider the MC of the page to be very high or highest quality when it is created with a high degree of time and effort, and in particular, expertise, talent, and skill—this may provide evidence for the E-A-T of the page”

Content that is vague and lacking in detail is unlikely to perform well, especially if other sites produce more comprehensive and useful pieces of content.

We’ve excluded questions that specifically tackle the E-A-T of the content creators in this section, with the aim to focus on evaluating the usefulness of existing content.

Not every business will be, or have industry experts producing content, but that shouldn’t detract from assessing the quality of content that is actually being produced.

Google states that smaller businesses are likely to have a smaller web presence and a lack of external evidence of E-A-T is not an indication of low page quality.

Under section 2.6.5,What to Do When You Find No Reputation Information, Google writes:

“Frequently,you will find little or no information about the reputation of a website for a small organization. This is not indicative of positive or negative reputation.Many small, local businesses or community organizations have a small “web presence” and rely on word of mouth, not online reviews. For these smaller businesses and organizations,lack of reputation should not be considered an indication of low page quality.”

However, for“Your Money or Your Life” (YMYL) pages, such as medical, or financial information websites, the reputation of the website and/or the individual content creator is just as important as the potential usefulness of content.

Evaluating the E-A-T of a website and/or content creator will be addressed in the topics “Would you trust information from this website’s authors?” and “Would you trust information from this website?”.

Does the [PAGE TYPE] on this site have obvious errors?

This topic is about identifying errors on a website, with a focus on recognising unmaintained and/or defaced pages.

“Unmaintained websites should be rated Lowest if they fail to achieve their purposedue to the lack of maintenance.”

“Unmaintained websites may also become hacked, defaced, or spammed with a large amount of distracting and unhelpful content.These pages should also berated Lowestbecause they fail to accomplish their original purpose.”

Although not explicitly mentioned in Google’s Search Quality Evaluator Guidelines, other signs of unmaintained websites are broken links and images as well as obvious missing blocks of main content.

If a website has a large number of glaring errors, it could compromise user experience and suggests the website may be insecure. Failing this topic indicates a website will likely struggle to perform well in search results.

Does the [PAGE TYPE] contain excessive adverts, or pop-ups?

Over the last couple of years, Google has been increasingly targeting websites that excessively contain adverts and/or block users’ access to content with pop-ups.

Under section 7.2.7,Obstructed or Inaccessible Main Content, Google writes:

“MC cannot be used if it is obstructed or inaccessible due to Ads, SC, or interstitial pages. If you are not able to access the MC, please use the Lowest rating”.

Google also suggests ads don’t need to fully obstruct a page to be distracting to receive a “low” rating. Under section 6.4, Distracting Ads/SC, Google writes:

“…some Ads, SC, or interstitial pages (i.e., pages displayed before or after the content you are expecting) make it difficult to use the MC. Pages with Ads, SC, or other features that distract from or interrupt the use of the MC should be given a Low rating”.

Whilst Google does state that ads can contribute to good user experience, they should be used in moderation and not negatively impact the experience of consuming main content.

Sites that contain excessive adverts and obstructive pop-ups run the risk of penalisation, which will impact online visibility.

This topic is particularly pertinent for mobile devices where screen space is limited.

The questions in our checklist for this topic also reflect Google’s firm stance on excessive advertisements. Any “Failed” question in our checklist will fail this entire topic.

Is [PAGE TYPE] deceptive?

Earlier, we mentioned how misleading page titles can be a means of deception, but Google cites other forms of deception.

For example, under section 7.6.2,Deceptive Page Design, Google states the following are forms of deception:

Pages that disguise Ads as MC. Actual MC may be minimal or created to encourage users to click on the Ads.

Pages that disguise Ads as website navigation links.

Pages where the MC is not usable or visible.

Any page designed to trick users into clicking on links, which may be Ads or other links intended to serve the needs of the website rather than to the benefit of the user.

Pages that are misleading or attempt to cause harm in some way are deemed the “lowest” quality of page and will negatively impact the perception of the organisation and hurt online visibility.

Is there adequate information about the business and/or content creators?

Google wants websites to provide adequate information about a business and/or authors.

Under section 2.5.3,Finding About Us, Contact Information, and Customer Service Information, Google writes:

“There are many reasons that users might have for contacting a website, from reporting problems such as broken pages, to asking for content removal. Many websites offer multiple ways for users to contact the website: email addresses, phone numbers, physical addresses, web contact forms”

Google adds that contact information is particularly important for websites that handle money.

“Contact information and customer service information are extremely important for websites that handle money, such as stores, banks, credit card companies, etc”

When it comes to individual content creators, Google doesn’t specifically mention that authors should have biography pages on a website. Google also state the E-A-T of authors should be primarily judged on external evidence; however, there is sound logic in providing an option to view biography pages on a website that demonstrate the E-A-T of authors.

“Low quality pages often lack an appropriate level of E-A-T for the purpose of the page. Here are some examples:

The creator of the MC does not have adequate expertise in the topic of the MC, e.g. a tax form instruction video made by someone with no clear expertise in tax preparation.

The website is not an authoritative source for the topic of the page, e.g. tax information on a cooking website.”

Ultimately, users want to be able to contact and learn more about an organisation/content creator. A website that provides adequate and reliable information that demonstrates E-A-T will foster trust and credibility.

Would you trust information from this website?

Google is emphasising the importance of websites and organisations having credible external reputation.

Google has stated they will trust external sources of reputation information over internal information.

Under section 2.6,Reputation of the Website or Creator of the Main Content, Google writes:

“…for Page Quality rating,you must also look for outside, independent reputation information about the website.When the website says one thing about itself,but reputable external sources disagree with what the website says, trust the external sources.”

Google wants to evaluate the external reputation information of an organisation using credible third party sources like external news articles, blog posts, or even Wikipedia pages.

Under section 2.6.2,Sources of Reputation Information, Google writes:

“Look for information written by a person, not statistics or other machine-compiled information.News articles, Wikipedia articles, blog posts, magazine articles, forum discussions, and ratings from independent organizations can all be sources of reputation information.Look for independent, credible sources of information.”

By identifying various sources of external reputation information, Google can better evaluate E-A-T.

Content that is written by websites/authors who meet a high level of E-A-T is deemed to be of the “highest quality” and in theory should perform better in search results. This is particularly important for YMYL websites/topics, such as health and safety, finance and news and current events.

“information about or services related to research or purchase of goods/services, particularly webpages that allow people to make purchases online”

Under section 5.2,Very Positive Reputation, Google writes:

“For shopping pages,experts could include people who have used the store’s website to make purchases;whereas for medical advice pages, experts should be people or organizations with appropriate medical expertise or accreditation”.

This suggests customer reviews for an organisation’s products/services can be used as a valid source of external reputation information for ecommerce and B2B websites.

As part of our checklist, we’ve included questions to evaluate online reviews for a brand, which could also encompass specific review information on a brand’s products/services.

Would you trust information from this website’s authors?

Like evaluating the E-A-T of a website/organisation, Google also wants to evaluate external reputation information for individual content creators.

Under section 9.2, Reputation and E-A-T: Website or the Creators of the Main Content?, Google writes:

“You must consider the reputation and E-A-T ofboth the website and the creators of the MCin order to assign a Page Quality rating.”

“The reputation and E-A-T of the creators of the MC is extremely importantwhen a website has different authors or content creators on different pages.”

Sometimes a website/organisation will be creating content and other times it will be individual content creators. Either way, evaluating the E-A-T for both types of content creators is important can be equally important.

Does the design of the website make it difficult to use?

This topic focuses on the functionality of the website and in turn, user experience.

Under section 7.2.3,Lowest Quality Main Content, Google writes:

“The Lowest rating should also apply to pages where users cannot benefit from the MC, for example:

Informational pages with demonstrably inaccurate MC.

The MC is so difficult to read, watch, or use, that it takes great effort to understand and use the page.

Broken functionality of the page due to lack of skill in construction, poor design, or lack of maintenance.”

The design of a website plays a pivotal role in functionality and since Google rightfully views bad functionality with a grade of “lowest rating”, the design of a website deserves its own topic.

A website with a good design and functionality will contribute to high quality pages and in theory convert better and potentially perform better in search results.

Would you trust this website with personal details?

This topic is the culmination of all the previous questions. If the overarching results are positive then users could be more likely to hand over personal information, such as credit card details and email addresses, which is an objective for most businesses.

There will be an option to select “TRUE”, “FALSE”, or “OKAY” for each question in the checklist.

Necessary

The importance (necessity) of each question will determine whether a section will “Pass”, or “Fail”. In the screenshot above, the question“Is it clear and easy to understand what the website offers?” is a necessity to pass and because the answer is “FALSE”, the section has “FAILED”.

You can view the necessity of each question by unhiding columns C-E but these additional columns can be confusing for people who don’t understand what the sheet is doing, so I would keep them hidden most of the time.

There are also two additional columns, “Ideal answer” and “Achieves ideal answer”

Ideal answer

“Ideal answer” determines what the “ideal” statement should be for each question.

In some cases you want the statement to be “TRUE”. In other cases, you want the statement to be “FALSE”.

For instance, we want the statement to be “TRUE” for the question “Is it clear and easy to understand what the website offers?”.

However, the “ideal” answer for the question “Is the content too short, insubstantial or unhelpfully vague?” would be “FALSE”. “FALSE” states that the content is in fact not short, insubstantial, or unhelpfully vague.

Achieves ideal answer

Using the above screenshot, we know the “Ideal answer” for the question “Is it clear and easy to understand what the website offers?” is “TRUE”“.

However, because the answer given is “FALSE”, the ideal answer has not been achieved; therefore, “FALSE” is the statement given for“Achieves ideal answer”.

This allows us to see all “Passed” and “Failed” sections of the checklist located in the“Passed and Failed section” tab.

We’ve also added a description and/or instructions and references column in the checklist questions tab to help explain questions, provide instructions and add context. Feel free to add to, or iterate the content found in these columns.

You’ll also find references of“[ADD BRAND]”in the “Checklist questions” tab. You’ll need to update these references to reflect the organisation the checklist is being run against.

The tab“Website content”concatenates example URLs and example titles to help make this checklist a little more efficient to use.

The easiest way to see how all of these fields work is just to fill them out with information from your brands first, and then notice how the questions have changed as you go through the checklist.

As we’ve mentioned, our Search Quality Evaluator Checklist can be used by a range of people.

SEO professionals: filling out this checklist is a great way to evaluate the UX and website quality of a client. It will allow you to pinpoint strengths and weaknesses and help you deliver priority recommendations.

Internal stakeholders: the checklist can be used by internal stakeholders to go through before publishing new content, website features or most things that could impact the UX and content quality. The checklist could also be filled out by internal stakeholders as a way of getting them on board with SEO priorities.

The public: if you want to get a true understanding of how users perceive your website and brand, then why not ask your potential customers?

Members of the publicNEED TOfill out allsitewide questions, but you shouldsplit page typequestions to avoid a horribly long survey that they’ll give lazy answers to. For instance, get a respondent to only answer questions for the homepage, blog pages, or product pages. Distilled’s originalPanda Surveypost contains some advice and guidelines on running a survey, including collecting data from respondents.