Netflix’s rise to being the world’s primary media streaming service was no fluke. It was based on a complex recipe of data manipulation and emotion that means the company knows what you want to watch even before you know yourself.

It is Netflix’s secret sauce of algorithms, big data, and gut instinct that fuel this unstoppable growth. It’s this secret sauce that allows Netflix to not just consistently recommend content that users will (likely) love, but also to fund the creation of that content, confident that it will be a success.

Incredible Amounts of Big Data

It’s no surprise that big data plays a big part in Netflix’s ability to recommend and fund the right content. What is surprising, however, is the kind of data and amount of data that Netflix tracks every time you use the service.

“Each time a member starts to watch a movie or TV episode, a ‘view’ is created in our data systems and a collection of events describing that view is gathered.”

As part of this process, Netflix tracks your “entire viewing history for as long as [you] are subscribed”. The system “gathers periodic signals throughout each view to determine whether a member is or isn’t still watching”. It also tracks your searches, ratings, geo-location data, device information, browsing behavior, time of day/week that you’re watching, when you decide to ditch a show, to pause, and to fast-forward.

With millions of Netflix users streaming billions of hours of content each month, the amount of data the company collects is bewildering. This data is massively important to the success of the company.

“75 percent of users select movies based on the company’s recommendations, and Netflix wants to make that number even higher.”

This viewer data is huge, and it’s imperative to why the service can be so addictive. Combined with the huge range of data stored about each show, it becomes hard to disagree with David Carr’s theory that “Netflix is commissioning original content because it knows what people want before they do” (emphasis my own).

Ever-Improving Algorithms

On its own, data is of little use. As Jason Gilbert wrote; “[Netflix’s] success is based on how well it’s able to choose programming that its viewers like while still being profitable.”

“[The company has developed] several algorithms, each optimized for a different purpose. In a broad sense, most of our algorithms are based on the assumption that similar viewing patterns represent similar user tastes. We can use the behavior of similar users to infer your preferences.”

This focus on viewing patterns is proving far more reliable than looking primarily at the rating you give to a show.

The company is constantly running large numbers of A/B tests (allowing user experience and algorithm changes to be rolled out and tested on small sub-sets of users) to iteratively improve each of those algorithms. According to Amatriain these tests “let us try radical ideas or test many approaches at the same time”. The primary aim is almost always to improve “member engagement (e.g. hours of play) and retention”.

Gut Instinct

“The abundance of source data, measurements and associated experiments allow us to operate a data-driven organization. Netflix has embedded this approach into its culture since the company was founded”.

The idea of shows being manufactured and recommended based solely on data is somewhat disturbing. But the TV industry has always relied heavily on data (often in the form of focus groups and viewer numbers). However, Netflix is taking this quite a few steps further.

That being said, Joris Evers, the company’s director of global corporate communications wanted to ease users’ minds. He told the New York Times:

“We don’t get super-involved on the creative side…We hire the right people and give the freedom and budget to do good work. That means that when Seth Rogen and Kristen Wiig are announced as special guests on coming episodes of Arrested Development it is not because a statistical analysis told Netflix to do so.”

In other words, the value of big data and algorithms informs Netflix’s decision rather than dictates them. Creative ideas for movies to fund, and shows to license will come thick and fast. Those that feel good, will be subjected to the data. If it looks as if a large enough section of Netflix’s users will be interested, and the decision makers’ gut feeling says the show will be a hit, it’s given a thumbs-up and a large check.

This Recipe Seems to Work

This mix of data, continually improving algorithms, and gut instinct seems to be working for Netflix. So much so, in fact, that the company has the confidence to fund entire series of shows before releasing a pilot episode. Most other broadcasters work the opposite way.

Producers and directors can pitch creative ideas to Netflix. If the big data and gut feeling add up, and suggest that the costs can be recouped in terms of new subscribers gained and increased retention, Netflix is able to go all-in. House of Cards is one example, where the company invested $100 million in two seasons without even seeing a pilot episode. And it’s why 2016 will see Netflix producing more original content than most other broadcasters do in several years.

This would not be possible if Netflix was not able to be unnervingly reliable at understanding and predicting what you (or at least most people) would love to watch. Before you even know yourself.

Well of course Netflix knows. It suggests that because I've watched Star Trek: The Next Generation that I may also enjoy watching Murder, She Wrote. We all know these two shows are almost identical. If I enjoy one then surely I'll enjoy the other. ;-)