Swapped a costly and time-consuming home-grown infrastructure for the scalability of Google BigQuery

Used BigQuery to help developers analyze vast amounts of data in near real-time so they can manage their applications more effectively

What they achieved

Bring new products and services to market nearly four times faster

Reduce time to run complex queries on large data sets from 30 minutes to 20 seconds

Shorten the amount of time spent to maintain their data analysis infrastructure by up to 40%

React to client needs faster

About Google BigQuery

Google BigQuery is a web service that enables companies to analyze massive datasets – up to billions of rows in seconds – using Google’s infrastructure. Scalable and easy to use, BigQuery lets developers and businesses tap into powerful data analytics on demand using familiar SQL query language. For more information visit cloud.google.com/bigquery

Claritics is a Mountain View, California-based social analytics company with 15 employees. Its web-based applications help social and mobile game developers, advertisers and media companies gain real-time insights into the behavior of game players and app users. These insights help shape key game and app design, optimize marketing effectiveness and increase application revenue.

Challenge

Claritics needed to help developers analyze vast amounts of data in order to make their games and applications more effective. The company’s original plan included setting up clusters of Hadoop servers to process the data, but the system was time-consuming and required a specialized staff to maintain it.

“Getting data into the system was cumbersome,” says Claritics CEO Raj Pai. “Once the data was in, running queries – asking simple questions and getting responses – took a lot of time. Complex queries on large datasets could take more than 30 minutes.”

As the company continued to analyze growing amounts of data, the Hadoop system required continual fine-tuning to expand its storage capacity. The infrastructure would also have to grow, meaning a hefty expense. Pai knew he needed a faster, more scalable data processing tool.

Solution

Google BigQuery is a web service that enables companies to analyze massive datasets of up to billions of rows in seconds using Google’s vast data processing infrastructure. Pai heard about the service at an event for Google developers in May 2010, and his company began using it that same summer.

“Google BigQuery is saving us time and resources. Since we don’t have to worry about setting up machines as we bring more clients onboard, we expect it will save us a lot of money as well.”

Raj Pai CEO, Claritics

BigQuery’s simple, SQL-like query language doesn’t require complex technology or specialized personnel. Claritics uses the service as a fast analytics database for its applications, pumping in large volumes of data – millions of gaming events – every day. Since the system scales automatically, Pai and his team don’t have to worry about fine-tuning or adding hardware as data volumes increase.

Claritics uses Google BigQuery to help developers:

Assess the effectiveness of user acquisition campaigns

Deepen game engagement

Find more effective ways to monetize their games through ads or virtual goods sales

Identify the most active users and entice them to introduce new players to games

“Rapid experimentation and testing of new games is critical for our publishers,” Pai says. “They look at user reach, retention and revenue metrics very carefully to see if a game or app is gaining traction before investing too heavily in it. BigQuery helps us provide an in-depth level of gaming analytics in near real-time, which is essential for our customers.”

Results

Google BigQuery has freed Pai and his team from infrastructure requirements, saving up to 40% of their time. In contrast with Hadoop’s lengthy processing time, running complex queries on datasets hundreds of gigabytes in size takes as little as 20 seconds using BigQuery, allowing Claritics to process terabytes of critical user data each day.

With Google BigQuery, Claritics is also able to innovate more rapidly. Since Pai’s team can run queries in mere seconds, they are able to conduct investigative ad hoc analysis to spot emerging trends in the data. This enables them to bring new analytics apps to market nearly four times faster than before.

“We can use the BigQuery infrastructure to create advanced queries and build these really quickly from a prototyping perspective,” he says. “This gives us more flexibility as we look ahead to developing new analytics apps. Google BigQuery is also saving us time and resources. Since we don’t have to worry about setting up machines as we bring more clients onboard, we expect it will save us a lot of money as well.”

The company also has the flexibility to roll out advanced analysis faster as developers’ games and apps evolve. “Using the big-data analytics infrastructure from Google BigQuery has significant time-to-market and performance advantages,” Pai says.