Lean split testing with Nginx and Mixpanel in 5 minutes

Here’s a dead simple way to set up a split test without using a fancy application server or Rails application.

Let’s say you’re starting out with a new application and you want to test a few versions of the landing page. In the truly lean fashion, you haven’t built any dynamic application backend yet. All you’ve got is two versions of the homepage in static html, using something like MailChimp on the backend to capture your signups.

Take your two versions and place them in a directory, along with whatever assets (css/js/images) go along with them:

public_html/welcome.html

public_html/hello.html

Let’s also assume that these pages contain some type of signup form that redirects back to:

public_html/thanks.html

That way we’ll know the goal is completed.

Notice that I’ve named them in user friendly ways. This will become important as we’ll have our webserver (nginx) redirect to those pages, so users will see them in the url. Why nginx? Because apache is a horrible nightmare to configure.

Now sign up for mixpanel.com and follow their instructions to place a snippet of their javascript into those two pages. Additionally, add an event to the bottom of each page to identify it:

welcome.html:
[cc lang="html"][/cc]

hello.html:
[cc lang="html"][/cc]

thanks.html:
[cc lang="html"][/cc]

Now, we’ll set up nginx to redirect to one of these pages “randomly”. Since there is no true random capability built in, we’re going to use the last digit of the user’s ip:

Now hit your website. If your ip is even, you’ll see the first page, and if it’s odd, you’ll see the second page.

Go to Mixpanel, and create two Funnels. One that starts with your welcome event, and one that starts with the hello event, and the second step in each funnel is the Signup Completed event. Now you have a funnel analysis for each page, and you can compare each to see how it performs.