The manual experiment creation wizard gives you full control over your A/B tests, allowing you to customize every aspect of your store listing experiments. This guide walks you through the complete process of creating a custom experiment from start to finish.
Manual experiments let you create custom A/B tests by hand-picking the assets you want to test, setting your own test parameters, and defining specific hypotheses. Unlike AI-generated experiments, manual experiments give you complete creative control over what gets tested and how.
Choose manual experiments when you:
Have specific design assets you want to test
Want to test a particular hypothesis or theory
Need precise control over experiment parameters
Are testing screenshot arrangements
Want to upload your own custom assets
Navigate to your app's dashboard
Click the "Create" tab in the main navigation
Select "Manual Experiment" from the options
Choose which type of asset you want to test:
App Icon (IC): Test different icon designs
Feature Graphic (FG): Test hero images and promotional graphics
Short Description (SD): Test different promotional text
Screenshots: Test screenshot arrangements and order
Your selection determines which options appear in the following steps.
Select the market and language combination where you want to run the experiment:
Use the dropdown to select a locale (e.g., "en-US" for United States English)
The system shows your current store listing for that locale
Only one locale per experiment - create multiple experiments for multi-market testing
Enter a clear hypothesis that explains what you're testing and why:
Good hypothesis: "Testing a more colorful icon to increase visibility in search results"
Poor hypothesis: "New icon test"
Your hypothesis helps organize experiments and makes it easier to analyze results later.
Set up the key parameters for your experiment:
Duration: How many days the experiment should run (minimum 7 days recommended)
Priority: Determines execution order when multiple experiments are queued
Traffic Allocation: Percentage of users who see each variant (typically 50/50)
CSL Assignment: Choose which Custom Store Listing to use (if applicable)
Add the assets you want to test:
Upload images directly from your computer
Select from previously uploaded assets in your library
Ensure assets meet Google Play requirements (dimensions, file size, format)
Add at least one variant to compare against the control (current live asset)
Before submitting:
Review all experiment details in the summary screen
Verify assets are correctly assigned
Check that settings match your testing goals
Click "Create Experiment" to add it to your backlog
Once created, your experiment appears in the backlog with "Pending Review" status. From here you can:
Edit experiment details before deployment
Adjust priority relative to other experiments
Submit for review and approval
Deploy to Google Play Console when ready
Test one variable at a time: Don't change multiple elements in a single experiment
Run for adequate duration: Minimum 7-14 days for statistical significance
Use clear naming: Descriptive hypotheses make results easier to interpret
Check technical requirements: Verify assets meet Google Play specifications before uploading
Plan your queue: Use priorities to organize multiple experiments logically
Issue: Asset upload fails
Solution: Check file format (PNG/JPG), dimensions, and file size limits
Issue: Can't select a locale
Solution: Ensure the locale exists in your Google Play Console store listings
Issue: Experiment not deploying
Solution: Verify higher priority experiments aren't blocking the queue
Understanding Asset Types - Learn about all 7 asset types (IC, FG, SD, LD, SS, PV, MULTI_ASSET)
Experiment Settings by Asset Type - Configure defaults for different assets
Manual Image Upload - Best practices for uploading custom assets
Multi-Locale Testing - Running experiments across multiple markets
