Experiment with promotion strategies using Firebase Predictions

With Firebase Predictions, you can optimize your in-app promotions for each
of your users based on the likelihood of the user making an in-app purchase. For
example, you can promote your more expensive "premium" bundle to users likely to
spend, and promote your less expensive basic bundle to other users.

Then, using Firebase A/B Testing, you can conduct an experiment to see how
this prediction-based strategy affects in-app purchase revenue compared to
simply always promoting the basic bundle or always offering the premium bundle.

Before you begin

Before you can start using predictions to determine your app's promotion
strategy, you must be using Google Analytics for Firebase in your
app. In particular, you must:

Explicitly log spending-related events that are not automatically collected,
such as ecommerce_purchase. (Firebase automatically logs the
in_app_purchase event for in-app purchases processed by the App Store and
Play Store.) You should also generally log any other events that are relevant
to your app, to maximize the data available to make classifications.

Make your promotions remotely configurable

First, set up your app to promote either your basic or premium bundle depending
on the value of a Remote Config parameter. In the steps that follow, you
will configure Predictions and A/B Testing to set the parameter remotely
based on various criteria, but for now, your app will always promote the basic
bundle.

For example:

Import the Analytics and Remote Config SDKs:

iOS (Swift)

Add the SDKs to your Podfile:

pod 'Firebase/Analytics'
pod 'Firebase/RemoteConfig'

Then, import them:

import Firebase

Android

Initialize Remote Config and set the default value of the
promoted_bundle variable to basic. Later, you will
set up Predictions and A/B Testing to set this variable remotely, but
by setting a default value in your app, you ensure that your app functions
correctly before remotely-configured variables are retrieved.

Kotlin

Promote the bundle specified by the value of promoted_bundle.
Until you set up Predictions, this value will always be
basic.

iOS (Swift)

self.remoteConfig.fetch() { (status, error) -> Void in
if status == .success {
self.remoteConfig.activateFetched()
}
// Act on the retrieved parameters
// Set the bundle to promote based on parameters retrieved with Remote
// Config. This depends entirely on your app, but, for example, you might
// retrieve and use image assets based on the specified bundle name.
self.promotedBundle = self.remoteConfig.getString("promoted_bundle")
// ...
}

Java

mConfig.fetch(CACHE_EXPIRATION)
.addOnCompleteListener(this, new OnCompleteListener<Void>() {
@Override
public void onComplete(@NonNull Task<Void> task) {
if (task.isSuccessful()) {
mConfig.activateFetched();
}
// Act on the retrieved parameters
// Set the bundle to promote based on parameters retrieved with
// Remote Config. This depends entirely on your app, but for
// example, you might retrieve and use image assets based on the
// specified bundle name.
mPromotedBundle = mConfig.getString("promoted_bundle");
// ...
}
});

Kotlin

config.fetch(CACHE_EXPIRATION)
.addOnCompleteListener(this) { task ->
if (task.isSuccessful) {
config.activateFetched()
}
// Act on the retrieved parameters
// Set the bundle to promote based on parameters retrieved with
// Remote Config. This depends entirely on your app, but for
// example, you might retrieve and use image assets based on the
// specified bundle name.
promotedBundle = config.getString("promoted_bundle")
// ...
}

At this point, you can deploy your app to the App Store or Play Store. Users
will get the default promotions behavior (always promote the basic bundle), but
because promotions are controlled by a remotely configurable variable, you can
change and experiment with promotion strategies using only the
Firebase console, without having to push app updates to your users.

Start predicting user spending

In the Firebase console, open the Predictions section. If you
haven't already agreed to the Predictions terms of service, do so.

After you have agreed to the terms of service, Predictions will be
enabled for your project. The Predictions section of the
Firebase console allows you to define custom predictions; however, for
spending predictions, you can make use of the built-in spend and
not_spend predictions, which use an aggregation of Analytics events to
predict whether a user will make an in-app purchase or an ecommerce purchase
(subscription purchases are not currently considered in this prediction). These
predictions will become available after you enable Predictions and have
logged a few weeks' worth of spending events.

Create a promotions strategy experiment

Now create an A/B Testing experiment that tests the effect of the
prediction-based promotions strategy on revenue compared to always promoting the
basic bundle and always promoting the premium bundle.

It is beneficial for an experiment to have three variants—a control group,
a group that unconditionally sets a parameter, and a prediction-based
group—so the experiment can determine whether the prediction-based
approach has a greater benefit than from simply applying the treatment to
everyone. However, an experiment with three variants requires more participants
for enough data to be collected. If your app doesn't have enough users to
support running an experiment with three variants, it can still be okay to have
an experiment with two variants, omitting the experiment where the treatment is
applied to everyone.

Choose your app from the list and specify how many of your users you
want to include in the experiment. You can also choose to exclude
certain user categories, such as high spenders, from the experiment.

Choose Purchase revenue from the list of goal metrics, and
select any additional metrics you want to track, such as user
engagement, retention, app removals, and ad clicks.

Define three variants:

Always promote the basic bundle (control group)

Always promote the premium bundle

Promote the basic or premium bundle depending on the spending
prediction

For the Control group, create a promoted_bundle parameter and set
it to (no change). Users assigned to the control group will get the
default behavior, which is to always see the basic promotion.

For the Always promote premium variant, set promoted_bundle
parameter to premium. Users assigned to this variant always see the
premium promotion.

For the Promote premium to likely spenders variant, set
promoted_bundle parameter to premium and set the user subset to
Predict: spend (medium risk tolerance). Users assigned to this
variant see the premium promotion if they are predicted to spend.

Start the experiment and let it run for several days or more, until
A/B Testing declares a leader. If the experiment cannot determine a leader,
you might need to expand the experiment to more users.

Roll out the winning variant to all users

After A/B Testing has collected enough information to declare a
leader—in this case, the variant that maximized purchase revenue—you
can decide whether to roll out the winning variant (or another variant) to all
of your users.

In the A/B Testing section of the Firebase console, open the details
view of the completed experiment. From this view, you can see how each variant
performed according to your goal metric and any secondary metrics you selected.
With this information, you can decide whether to roll out the leading variant or
another variant.

To roll out a variant to all users, click
more_vert > Roll out the leader on the
experiment's details page.