The name, A / B testing, speaks for itself: it experiments with graphics and text. Upload multiple creative materials, select a percentage (%) of users for each variation, and wait for the results. With a lot of downloads, 10-14 days are enough for the test. Sometimes, however, the result of an experiment becomes obvious during the first week. To get a more accurate picture, do not limit yourself to A / B, but also B / A, A / B / B, and other tests. Using these methods, you can check the reliability of the first results. Try not to change anything in the application during the experiment since that might affect the indicators, and you won't be able to figure out if the experimental features or other factors were responsible for that.

Still, don't trust the tests blindly, be sure to analyze indicators before and after the experiment. There were cases where creative graphics showed good data when tested but, when applied, conversion rates dropped.

On Google Play, you can conduct tests for free in Google Play Console, just go to the Experiments section. On AppStore, however, this is not possible, and you'll have to resort to third-party services. If you are unable to use third-party services for AppStore, you can run a sequential test, that is update creative materials one by one and analyze the metrics. But this option is only ok if you are not scared to make the conversion worse, and if the result is negative you will quickly restore original values.

We wouldn't recommend testing creative materials for one platform and then using the winner on another platform. Google Play and AppStore have different audiences and different perceptions of graphics. Thus, a successful creative material for one platform may fail on another. Once we applied a new icon for Kid-E-Cats on Google Play and decided to do the same on AppStore because it showed excellent results during the test. As a result, conversion rates on AppStore began to drop:

We had to urgently reassemble the build and put the old icon back.

Do not forget to update your old projects because you can attract new users and give a second life to the game. In our old Fixies we conducted a successful experiment: updated the icon and screenshots and got an excellent increase in performance. What's more, thanks to the update, the project got into a very relevant list, and look what happened to the downloads:

However, here's the case with YooHoo: Doctor. We never took into consideration icons with secondary characters, thinking that everyone was interested in the core character, but still decided to test them just in case. Surprisingly, this gave an excellent result, see how downloads jumped:

But sometimes an application takes such a strong position in a store that new creative materials lose to the existing ones. It was the case with Robocar Poli, with the old icon winning all the tests. We made about 15 new icons, tested them for a long time but had to give up and leave the old one be.

If your application is localized into other languages, be sure to prepare creative graphics in those languages and test them too. Cultures and interests vary from country to country, and what scores well in one country may not be at all successful in another country.

And, of course, test out holiday creative materials (you can read about this in our other article, "Holiday updates in games"). Actually, users sometimes like the holiday graphics so much that even a couple of weeks after the holiday it performs better than the regular one. We had a case like that: after a successful test, we implemented a New-Year icon for Kid-E-Cats. Soon, the holiday was over but according to the results of the experiment users still liked the new icon.

It's up to you what to do in a case like that: you can leave the festive icon but do not forget to watch the indicators.

Be sure to test everything and always: even if you are confident about your new creative materials, users may like an option that's not immediately obvious. Do not forget that trends and tastes of the audience are constantly changing - say, half a year ago one thing trended, today you can lose your installs and positions because of irrelevance. Experiments do not take much time and effort, but the right creative materials boost downloads and conversion, while giving a chance to get into the recommended list. And don't be afraid to fulfill your wildest ideas, they might work too!