Introduction to Mobile Analytics & Reporting

I recently spoke as a panelist at the MG West conference on mobile game analytics and after the panel was very surprised by how few companies and people in the industry seemed to know about analytics overall as well as how to properly execute an effective analytics program.

In this post I’d like to answer the first two of the following key analytics questions:

How should you think about analytics at a high level?

What should you be measuring?

What metrics do you need for a successful game?

In a future post, I’ll also address the third question from the above.

1. How should you think about analytics at a high level?

First of all, you should take the approach of trying to think about analytics from a Role and Application perspective. In other words, who is looking at the data (Role) and what are you using the analytics for (Application).

Let’s talk about some of the key applications for a moment first:

Audit: How well is the game doing and how should we think about the game?

User Acquisition (UA): What are the target metrics we’re shooting for and how and how much should we spend to acquire users?

Optimization: How do I uncover insights about how to make the game better and how do we do that?

Anti-Hacking: How can I detect cheaters or hackers in the game?

In a typical mobile game company setting this may break down as follows (simplified view below):

D1/7/30: Retention on Day 1, Day 7, and Day 30. This is computed by calculating the actual % of users that logged into the app on the specified day from day 0

% Loss: % of users that are considered inactive after not logging in after a specified # of days e.g., 7 days

Engagement:

Sessions per Day: # of sessions / DAU

Session length: Average amount of time spent by each user in a session

Session length by level

Stability:

Crashes per day:

Server: Number of server or server process crashes per day

Client: Number of client crashes per day

Crashes per 1K (or other # depending on your traffic) DAU

Top 3 crash types: You should try to characterize crashes and isolate where crashes are occurring.

Client: It’s typically more difficult to characterize a client crash but at the least you should send up a client dump to help developers try to figure out what happened.

Server: Server crashes are relatively easy to isolate and you should be able to tell even by the line number where problems occurred

Performance:

Load time for X by device type: One of the biggest sources of user loss is wait times for loading. After playing your game you should try to identify a few potential problem spots and opportunistically track load/wait times for those areas.

Device type: By characterizing load/wait times by device you can see if there are measurable differences in KPIs based on different wait times you will see on the various device types

UA:

From a UA perspective, we like to measure source effectiveness and do source/user attribution. Basically how effective is a $ I spend to acquire users? What is it getting me?

Marketing:

CPI: Cost per install

Effective CPI: CPI including organics, social, etc. that may reduce the overall cost for a user install

LTV (or LTV proxy)

Attribution

UDID/IDFA lists: You should always check to see if you actually got the users that your vendor claimed to have sent you

Is LTV – Effective CPI > 0?

3 Side Points:

For the majority of the top grossing games on iOS App Store the answer to the above should be: NO. Unfortunately, we still have a broken App ecosystem from Apple… but I digress

Many companies totally screw up LTV calculation and make forward looking projections where they project an overly optimistic LTV, overspend on UA, and in the end get screwed (*cough* *cough* GREE *cough* *cough*)

CPI in itself is not the full story as cheap CPI may come with poor traffic. Do not make the assumption that your overall LTV should be applied to any new source of cheap CPI traffic (*cough* *cough* Tapjoy bots *cough* *cough*)

3. Optimization:

This is the most custom part of building analytics for your game and will require a deep understanding of your game to determine what specific metrics are necessary. The overall objective is to tune/balanc:

Game Economy

User Progression e.g., how fast are users moving through the game in PVE, PVP, any other loops

User Activity e.g., what users are actually doing in your game… how much, how long?

Game Content/Balance

Monetization

Retention

Social/Viral

Here’s a typical list of stuff I came up with for some of my games:

Game Economy:

Sinks and sources of soft currency by level

Stack bar graph of each source

Item by popularity (last 3 days, overall)

Soft Currency

Hard Currency

User Progression:

By Level:

% of players

% of plays

Average amount of money

Highest amount of money

Average net worth (including item value)

Highest net worth

Average # of days

Average # of sessions

Average # of battles

By Loop (e.g., PVE)

Show average progression by Level

E.g., Level vs. Battle # (PVE progression) as a % of users and as a total user count

Show highest progression by Level

Average mastery level by enemy/map

User Activity:

Sessions:

Average sessions per day

Highest sessions per day

Average time per session

Sessions by Level:

Average sessions per day by level

Average time per session by level

Activity Type

% of activity type e.g., PVP vs. PVE vs. Other Loop

Game Content/Balance:

% character type owned

% win by character type

Show top 10 squad types (e.g., {archer, mage, knight}, etc.) by win %

Show top 5 character types by win %

Show bottom 5 character types by loss %

Monetization:

ARPDAU

ARPPDAU (Average revenue per paying DAU)

% of users who purchase

% of users who purchase by level

% of users who purchase by source

ARPU

1 day, 7 day, 30 day

LTV

Average $ amount spent

$ spent by level

First day buyer conversion %

Store

% first item purchased

% spend by category

Top 10 items purchased

Bottom 5 items purchased

Store heat map by clicks (where do users click the most on the store)

Retention:

Tutorial completion

1 day, 7 day, 14 day, 30 day

Funnel by tutorial and level

Social/Viral:

% FB connect

Average # of FB friends installed

by FB Connect users

Average # of FB posts (per user, per active user)

by FB Connect users

% of users who have FB liked

by FB Connect users

4. Anti-Hacking

All games can be hacked. Especially mobile games with primarily client-based architectures on Android that will be especially prone to hacks and cheats.

Here is a simple list of things you should track:

Revenue velocity change

If revenue by day decreases or increases by > X% (set X = 20%)

If user net worth > 10% in 1 day after level 5 alert and log

If # of sessions > 10 per day alert and log

If user increases > 1 level per day after level 10 then alert and log

That’s it for now… I’ll also answer the 3rd key question around typical heuristics to indicate a successful game in a future post.

UPDATED: Crash/performance analytics added to this post on 6/11/’13 based on feedback from Lukasz Twardowski. Also Lukasz makes a very relevant point that the term “analytics” is often used in our industry (and me too) where “reporting” is actually the more appropriate term. Thanks Lukasz!

Hey Joseph, thanks for this really brilliant article (I am not exagerating).
I’ve attended many industry events in recent months around the globe and the truth is that many (even very big) players in the market are not close to where they could or should be in terms data driven decision making. I think its just such a still jung, nacsent industry that its hard for everyone to keep up with the tremendous pace of change…think things need some time to evolve. cheers, tim

Joseph, I’ve been working on social mobile games for several years, attended dozens of workshops from Casual Connect to GDC 20XX, but your site has been the most informative, easy-to-understand breakdown of how social games tick and what to do to maximize returns. Thanks again and keep up the great work!

Nice one, goes along the lines of the Lean Analytics book, but you were able to list the actual metrics and how to measure them in a slightly clearer manner, thanks! It seems everybody is talking about this but very few are actually doing anything about it. 🙂

I’ve used a combination of tools depending on the game and company (typically there may be existing relationships or special deals in place).

For cost conscious guys who are more technical and know what they are doing, I recommend a combination of Leanplum + custom analytics basically filling in where you don’t have third party support. Have also used Kontagent and like what I’ve seen from Swrv. There are advantages/disadvantages for each platform.

I’ve also looked at Playnomics (which is slightly different) and like what they can do from a theoretical perspective but haven’t actually used it yet. Also, heard a lot of good things about Staq (acquired by Playhaven) but haven’t used it myself yet.

More

About Me

Business and life philosopher. Clash Royale Fan. Father of 3 crazy kids. Mobile gaming executive currently working at NBCUniversal Games & Digital Platforms group. All opinions expressed in any of the content found on this website are my own and do not represent the opinions of my employer. More.