Redirect variations and right clicks not working

Redirect variations and right clicks not working

We have an experiment running and we’re trying to understand the results. In so doing, we’ve discovered that if you right-click on a link to get to the experiment page, then the redirect (to the variation) does not seem to happen. Why would this be?

I’ve tried the following:* Bucketize my browser to ensure I’m in the variation (redirect experiment)* Right click on a link to open the test page in a new tab, and the redirect does not fire* Refresh the page (soft and hard) and still the page does not redirect (and no query params added)* Go back to the page where the link was, and click on it normally (it happens to load in a new tab via target=“_blank”), and it redirects to the variation correctly

Re: Redirect variations and right clicks not working

This is happening due to how Optimizely defines+checks for a new visitor and how sessionStorage behaves.

These are the steps Optimizely takes to define a new visitor:1. Check if the visitor has a optimizelyEndUserId cookie, if none is present, create one. 2. If the visitor is a new visitor, then drop a sessionStorage item so that Optimizely can continue to identify this visitor as a new visitor during this session.

When you open a new tab a new session is created. This means that the sessionStorage item is lost in the new tab because of the way sessionStorages are defined. From Mozilla's docs on sessionStorage:

"A page session lasts for as long as the browser is open and survives over page reloads and restores. Opening a page in a new tab or window will cause a new session to be initiated, which differs from how session cookies work."

However, the optimizelyEndUserId cookie remains in this new tab, so Optimizely sees the visitor as returning.

Re: Redirect variations and right clicks not working

The documentation from Mozilla and other places seem to differ slightly, and I it wasn't clear to me why from Mozilla's page there should be a difference between 1) clicking on link that would open in a new tab by virtue of a target="_blank" and 2) the user right clicking on such a link to force it to open in a new tab anyway. So I wrote a quick JS test to test / prove this, and indeed you're right. I'm still not clear on the rationale...

However, this will affect our test results, as I see it. What I don't know is by how much. Because currently we measure (new users only):

1) A goal for users who open a form (currently up a great 17%)

2) Goals for later down the funnel, where they could have gone back to a category page, right clicked to open in a new tab, and then not been in the experiment any more when they complete the other goals.

Am I correct in that analysis?

What does this mean?

1) In future, shoudl we run tests without the segment "New Users" only? I suppose we could use that for simpler single-page-and-goal type tests?

2) I should probably look back over past tests, and see if there are similar ambiguous or error prone results?

A/B testing is complex. Also, the difference / missed users will not be pushed to Google Analytics, as they're not being bucketized!

Many thanks for the help. If you could offer advice as above that would be great.

Re: Redirect variations and right clicks not working

The short-version of what is happening is that the new tab is actually a new window. Your browser just happens to make this new windows look just like a new tab, because "reasons".

Given the preponderance of "Private" and "Incognito" users (some of which forbid using sessionStorage, like iOS Safari where sessionStorage exists but has a max-size of 0, so you cannot put anything into it), anything that relies on sessionStorage is going to break. Since your site uses _blank, you will need to create a work-around for this.

My suggestion is to create a Custom Audience that lets user who meet the following conditions into it:

1- the optimizelyEndUserId cookie does not exist.

2- users who have a specific session cookie (e.g., opt_experimentId, where "experimentId" is the actual ID of the Experiment that you want to use this audience for).

#1- this would be truely new users on their first page hit.

#2- when your experiment runs, you are going to create this session cookie so that when a user reaches their second page, this cookie acts as their key to getting into the experiment.

If the number of cookies on your site is an issue (or you run lots of experiments targeting new users and don't want to create a "too many cookies" issue), instead of creating a specific cookie for each experiment, you could use a single cookie that contains a deliminted list of experimentIds in it, e.g. optNewUserExperiments. The experiment code would add the value "|experimentId" and your code in #2 would look for "|experimentId".