Questionable experiments

15 April 2013

Are there SEO benefits for using Google+? Many people, including those who have conducted experiments, seem to jump to conclusions, and assume a few things, when a controlled experiment on this issue is impossible. This piece outlines some of the questionable methodologies and assumptions that some experimenters employ, and casts doubt on the certainty with which those experimenters conclude their studies.

There has been a lot of buzz and noise surrounding the supposed benefits of using Google+ with regard to SEO (search engine optimization), based on the results of various experiments. I am not an SEO expert, but I have been around long enough to know that I don’t know a lot, and some of the recent findings touted in Google+ need to be treated with some caution. I cannot disprove the findings, and such is not my intention, but I am questioning the methodologies, assumptions, hypotheses and conclusions underlying some of the studies. Furthermore, it is suggested that there are perfectly reasonable alternative explanations, in which Google+ is not something special, and the authority of the author as opposed to that of the page is not yet an important factor.

One of the biggest problem with conducting experiments with Google search rankings is the near impossibility of making it controlled to any appreciable degree. Put it simply, there are too many variables that a conductor of an experiment cannot control. For starters, Google alters its algorithms on a daily, even hourly, basis: Google results of tomorrow may be different from today, and that of today different from yesterday, because Google changed something. Then, there are actions by other parties: new pages are created, existing pages are edited, and old pages are deleted; new links are created, existing links are edited, and old links are deleted. The experimenter does not and cannot know the changes that took place outside of his or her control during the period of experiment. In other words, there may be bigger and numerous variables in play than what the experimenter is varying. A properly controlled experiment will thus have to put the entire web as it is in aspic for the duration of the experiment. It would be easy to conclude myopically that because the ranking has changed, it must be due to the variable that the experimenter was testing, when in fact such is attributed to other factors. It’s like not seeing the whole forest move, because the experimenter was looking at a single tree. Any experiment with Google search results thus starts with an unknowable amount of uncertainties, or unknown unknowns. One way to mitigate this issue is to conduct a large-scale experiment, which is beyond the capacity of most individuals, and even that of many companies and organizations.

In setting up an experiment on the degree to which a Google+ profile as an author affects its interactions with external websites, controlling the experiment is of paramount importance. Yet, attaining such a level of control is well-nigh impossible, even if there were no fluctuations in the search algorithm. If the object of the experiment is, for example, to demonstrate that links or actions such as +1s or sharing from certain types of Google+ profile as author will have a more effect than others, then the linked pages must be essentially the same. They cannot be different: otherwise they become variables. The variable under investigation is the type of Google+ profile, such as their dates of creation, degree of engagement, number of posts, number of people circled and circling the profile, and relevance of the content. It should be clear, that some of these are quantitative, hence easier to control, but many of them are qualitative, and it becomes harder to assess. Because these are all variables, for an experiment to be rigorous, there must be hundreds of different Google+ profiles, each looking at a specific quantitative or qualitative variable. Even if it were possible to create numerous Google+ profiles and numerous files with almost the same file names, such as page_1, page_2, etc with exactly the same content, uploaded at exactly the same time, it would not be possible to control when Google crawls the page, and that could be an issue, because these pages are duplicates. Again, this crucial factor is out of the controls of the experimenter. Assuming, miraculously, that all pages are exactly the same save the file names and the Google+ profile they link back to as their author, that they are crawled exactly the same time, and that actions on Google+, such as sharing and +1ing, took place at the exactly the same time, would that be sufficient to conclude that whichever page is ultimately indexed has been indexed because of the action in the Google+ landscape? It is plausible to assume so, but without access to how Google goes about crawling and indexing, it has to remain at best a well-educated guess.

There seems to be an assumption that anything on Google+ intrinsically has to do with authorship, and authorship with the authority of the author, which will influence the organic search rankings. Is this assumption correct? This needs testing, though a controlled condition allowing such an experiment is well-nigh impossible to achieve. If the object of the experiment is state that authorship, as in a bi-directional link between the author’s Google+ profile and a web entity authored by that person, has a positive effect, then it will be necessary to set up two equal Google+ profiles, and link to or +1 pages, and see if there are any differences. The Google+ profiles must be exactly the same (history, posting habits, engagement, circles, etc), except that one has a bi-directional link with the page, and the other is someone else’s profile, i.e. not the author’s. Finding such a pair of Google+ profiles is virtually impossible, since it would require the experimenter to set up Google+ in a controlled environment, which Google+ cannot be in. Not only would the experimenter have to create two exactly same Google+ profiles, he or she will have to control all other profiles and pages that are in the profiles’ circles or have circled the profiles. In other words there needs to be hundreds, if not thousands, of profiles, and each must also be rigorously controlled. If there is a material difference between the two links, then it may be surmised, with due qualification, that a bi-directional authorship mark-up has an effect. If there are no appreciable differences, then it would be plausible to conclude that authorship does not influence rankings.

Perhaps there is a more fundamental question to be asked: is there anything special, qualitatively different, about a link from or an action on Google+, in comparison to other online presences, that affects the search engine rankings? If the object of the experiment is to find that Google+ is different from other forms of presences online, then the comparators must be a Google+ profile and something that is of equal strength as far as Google ranking is concerned. This is an impossible task, as such would effectively require a control over Google+, the complete Google algorithm as it stands, and setting up a web presence that would otherwise be equal to Google+.

In the preceding paragraphs, I have outlined three experiments that can be conducted to answer three different questions: (i) how much does author’s authority tied to a Google+ profile matter in relation to search engine rankings; (ii) does author’s authority tied to a Google+ profile exist in relation to search engine rankings; and (iii) is Google+ special? Should they be conducted in the order that I have listed? Probably not. However, some experimenters are too keen to do the experiment on the strength of the profile and author’s authority (i), blindly assuming that authorship is inherent in Google+ in affecting search engine rankings (ii), and as such it is different from other web presences (iii), when there needs to be more experiments to establish whether such assumptions are warranted in the first place.

Google+, so long as it is public, and can be crawled and indexed, is like other entities on the web. It follows that if a Google+ profile posts a number of times on a certain topic, the content on the profile, not the author, are seen as relevant to that topic. If people link to or share the posts, internally within the Google+ landscape as well as externally, for example, then the posts or the profile will be seen as authoritative. Should an authoritative site, whether a Google+ profile or a site or a blog, link to another site, that will be a boost for the site that has been linked. In that sense, there is nothing special about a link, so long as it is a PR-passing link, from or an action on a Google+ profile. If there were a link from or an equivalent action on another equally authoritative source as a Google+ profile, then would there not be the same positive effect as a Google+ profile? Perhaps Google+ is special, but I have not been persuaded by any study or claim thus far, and I cannot see how Google could treat Google+ more favourably than it would an equally strong site in organic search, as such action would be deemed unfair and abusive by its competitors and the regulators. I guess, and it is a pure and idle speculation, that authorial authority and credibility will be used by Google at some point in the future as a ranking factor, but I doubt it will be anything crude as some score on Google+: if the information is tied to Google+ and only Google has a privileged access, then using it as a ranking factor in organic search will be deemed unfair and abusive by Google’s competitors and regulators, but it would be suited for some form of personalized search, perhaps an advanced or refined version of SPYW.

For these reasons mentioned, I am very sceptical about studies that conclude a number of things without qualifications and nuances, based on a very small sample size. However, this does not mean that all experiments are entirely worthless or completely flawed, but the experimenter needs to be prudent. Those whose views and opinion I value tend to be circumspect in conclusion, rigorous in hypothesizing, clear and critical about their own assumptions, and open about the limits in the design and methodologies of their experiments. They are not afraid to say I don’t know. I am always wary of those who seem to be supremely confident about their pronouncements, expect others to follow them blindly or hand over money to them, when there are so many factors involved. Above all, the respectable experimenters and thinkers see the bigger picture, the complexity, and do not reduce something as complicated and fluid as Google search into a to-do list or a simple causation. Well-designed experiments can provide better approximations, which make the bigger picture that bit clearer, and webmasters will benefit from such experiments, but not from badly-designed experiments, which are likelier to give skewed and misleading, possibly outright wrong, results.

The foregoing should not be taken as a reason not to invest in Google+, or establishing authority and reputation as an author. Indeed, I would advise people to be very conscious about their interaction as an author in Google+, which may in the future reap benefits, but I would not expect there to be directly attributable SEO benefits today, or to cultivate a Google+ presence primarily for that reason. In the end, personal authority, reputation, credibility and trustworthiness are integral in Google+ and may be crucial in future when interacting with others on the web: they can hardly be gained or faked overnight, but will require proper nurturing.