Run A/B tests on your Store Listing

To help optimise your Store Listing on Google Play, you can run experiments to find the most effective graphics and localised text for your app.

For published apps, you can test variants against your current version to see which one performs best based on install data. Before setting up a test, review the best practices for running effective experiments.

Experiment types

For each app, you can run one global experiment or up to five localised experiments at the same time.

Using a global experiment, you can experiment with graphics in your app's default Store Listing language. You can include variants of your app's icon, feature graphic, screenshots and promo video.

If your app's Store Listing is only available in one language: Global experiments will be shown to all users.

If you've added any localised graphic assets in a specific language: users viewing your app in that language are excluded from your app's global experiments. For example, if your app's default language is English and it has a localised feature graphic in French, users viewing your app in French will be excluded from the experiment (even if you're testing your icon).

Using a localised experiment, you can experiment with your app's icon, feature graphic, screenshots, promo video and/or your app's descriptions in up to five languages. Experiment variants will only be shown to users viewing your app's Store Listing in the languages that you choose.

If your app's Store Listing is only available in one language, localised experiments will only be shown to users viewing your app in its default language.

Step 1: Create an experiment

You can create an experiment using your Play Console. When you're ready to review and apply results, you can use your Play Console or the Play Console app.

To apply a variant that outperformed your app's current version, select Apply winner. You can review the changes to your Store Listing before the changes go live.

To keep your current version, select Keep. This will update your Store Listing to use your current version and end the experiment.

If your experiment results in a tie, select Stop experiment.

Results

Once you select an experiment, you can view specific details about how each variant performed.

Variant

Definition & examples

Audience

% of users that see a variant of your experiment

Installs On Active Devices

# of devices that have been online at least once in the past 30 days and have your app installed

Scaled installs

# of installs during your experiment divided by audience share

For example, if you ran an experiment with two variants that used 90%/10% audience shares and the installs for each variant were A = 900 and B = 200, the scaled installs would be shown as A = 1000 (900/.9) and B = 2000 (200/0.1).

Performance

Estimated change in install performance compared to the current version. There is a 90% chance that the variant would perform within the displayed range over time. Since the range is based on a variant's performance, these numbers will vary during the experiment.

The average between your variant's high and low approximate change in install performance represents your variant's estimated change in performance.

Example: if one of your variants had a performance range of +5% to +15%, the most likely change in performance would be the middle number between the two, about +10%.

Performance will only be displayed once your experiment has enough data. In general, as an experiment has more time to run and collect data, a variant's performance range will become more narrow and accurate.

Scroll down to the 'Experiments' card. For more information on your app's experiments, tap View details. From the detailed view, tap on a variant to see the audience, current and scaled installs for that experiment.

To apply a variant that outperformed your app's current version, select Apply winner. You can review the changes to your Store Listing before the changes go live.

To keep your current version, select Keep current. This will update your Store Listing to use your current version and end the experiment.

Results

Once you select an experiment, you can view specific details about how each variant performed.

Variant

Definition & examples

Audience

% of users that see a variant of your experiment

Installs On Active Devices

# of devices that have been online at least once in the past 30 days and have your app installed

Scaled installs

# of installs during your experiment divided by audience share

For example, if you ran an experiment with two variants that used 90%/10% audience shares and the installs for each variant were A = 900 and B = 200, the scaled installs would be shown as A = 1000 (900/.9) and B = 2000 (200/0.1).

Performance

Estimated change in install performance compared to the current version. There is a 90% chance that the variant would perform within the displayed range over time. Since the range is based on a variant's performance, these numbers will vary during the experiment.

The average between your variant's high and low approximate change in install performance represents your variant's estimated change in performance.

Example: if one of your variants had a performance range of +5% to +15%, the most likely change in performance would be the middle number between the two, about +10%.

Performance will only be displayed once your experiment has enough data. In general, as an experiment has more time to run and collect data, a variant's performance range will become more narrow and accurate.