A/B Testing: The Post-Publishing Marathon Run

Hidden Escape : Secret Agent, Vincell Studios

A/B Testing: The Post-Publishing Marathon Run

Games go through a natural evolution cycle from ideation to launch. What starts as a germ of an idea finally becomes a product, an organic entity of its own. Most people think that once the game is released, you leave it to the high heavens to decide its fate. Not quite true. The wheels at a gaming studio never stop turning. Setting your game up on the Google Playstore or the iOS app store is a mini project unto itself. Then there are app store optimizations, and A/B testing plans to be considered. Why take all the effort, you ask? So that the game speaks to you when you browse through the store.   

But first, let’s step back and understand what the Play Store and iOS app store want and why?

  • App Icon: An icon is the first thing you’ll see as a player, and frankly, it helps to make a good first impression. It must reflect what the game is about. 
  • App Screenshots: There’s a guideline to follow while creating the screenshots. However, what’s most important is that the screenshots must communicate the user experience to the players. It must help you decide if these features and experiences are worth your time or not. 
  • App Promo Video: Although not mandatory, videos are an important way of showcasing what the game is about. Various studies suggest that players make their decisions in less than 5 seconds. 5 seconds is all we get for months of hard work! So we have to make the video count.  

Now that you know what the requirements are and how they help in optimization, it helps to have an A/B test plan in place. What is that you ask? Studios must create variations of all the visual elements -i.e. icons, screenshots, and videos, and run them as tests. This will help evaluate their impact on the ASO performance, and the game’s marketing success. 

We assure you this is not a random marketing gimmick, but certain creatives instantly hook the players, or at the very least make them curious. As research suggests the first 3 screenshots are the make or break for the game. We ran experiments on screenshots of one of our titles, Escape Games – Spy Agent on Google Playstore. Here’s what we found: 


Launch Screenshots:

The game was released with a set of more generic screenshots. The screenshots emphasized the action aspect of the game. They also introduced you to the various interesting characters in the game. However, there was no clear indication of the puzzles that were there in the game, and hence it failed to attract the attention of the puzzle heads. 

Variant A:


This new variant showcased the various obstacles and odds the players will face as they go through the game. It created a different sense of purpose and urgency for the narration of the game. 

This is our most successful experiment, and it showed a nearly 30% increase in downloads. That’s huge, right? 30% of users might have scrolled past our game, or would’ve never downloaded had we not experimented.  

Variant B: 


This variant focused on organizing the screenshots in a different order resulting in a different narrative option. It presented the game as an exploration and adventure journey. 

Though the Variant A stood out, both the experiments performed remarkably better than the first option.   We’ve understood that being complacent in the gaming business is not an option. What matters is to continue testing so we can make informed decisions. The App store optimization and A/B testing are not foolproof, it’s a mix between constant experiments and relentless research. This will help us analyze the game’s performance and help reach out to you better so that you can decide its fate.

Leave a Reply

Your email address will not be published.