Magoosh

How Magoosh uses in-product NPS to A/B test for customer happiness

Benefits of using Wootric in-product NPS surveys to optimize product:

An agile framework that can be replicated for testing future features and products.

“My advice would be to shift your thinking on NPS from a one-time transactional model to an ongoing and contextual model. You can speed up decision making and keep your pulse on customer happiness.”

Magoosh swears by Net Promoter Score. “NPS is one of the most important metrics we track,” says Peter Poer, who is responsible for product content at the online test prep company. “It has been a reliable leading indicator of growth in word-of-mouth referrals — our largest marketing channel.” High NPS means lots of happy students spreading the word, so when Magoosh noticed the metric drop, their immediate impulse was to address the issue causing the score to lag.

Because Magoosh only sent out an NPS survey once students were done using the platform, unfortunately, months would pass between the time that a student saw a new feature and the time she rated the product. How could Magoosh know whether their fix was going to improve customer satisfaction? Making a significant change without understanding how it could affect word of mouth marketing was a big risk. Magoosh needed a method for collecting NPS data while students were still studying so that they could do something they’d never attempted before — A/B test for NPS optimization.

The solution was to use Wootric to bring NPS inside Magoosh’s product and collect data in real time, right while the students were using it. The Wootric dashboard automatically segmented NPS data by student groups with the new product versus control, so Magoosh was able to A/B test and make a much faster decision to roll out the change. The result was a nine-point increase in NPS for the treatment group — which, of course, translates into happier students getting the word out to their friends. The best part is, Magoosh can now confidently optimize for referral while students are still studying, with a useful, agile and repeatable framework for testing future features and products.

CUSTOMER

Peter Poer, Director

GOAL

Optimize product quickly to address a drop in NPS.

OBSTACLE

Transactional model of NPS meant feedback would come too late to assess how feature changes would impact overall customer satisfaction.

RESULTS

Using Wootric in-product NPS surveys to A/B test for customer happiness led to:

Faster decision-making.

An agile framework that can be replicated for testing future features and products.