Run A/B tests (also known as split tests) on Amazon product content to optimize for conversions using the Manage Your Experiments tool. By running experiments to compare how shoppers interact with two versions of content, you can find out what resonates with customers and whether different details result in more views, click-throughs, and purchases. Experiment results indicate how each version of content performs along with the probability that the winning content is more effective in driving sales.
What is Manage Your Experiments?
Manage Your Experiments is a content optimization program that allows Amazon sellers to run A/B tests on Amazon product titles, images, descriptions, bullet points, and A+ Content to see if different versions are more likely to convert to sales.
How does Manage Your Experiments work?
Here’s an overview of the Amazon split testing process:
- Plan your experiment (decide what you want to test)
- From the Manage Your Experiments main screen, click the Create a New Experiment dropdown and select your desired experiment type
- Create an alternate version of the content (Version B) and start the experiment
- Review results to see if one version of the content performs better
- Publish the winning result and continue to experiment to further optimize content
To get started, sign up for a Professional selling plan and enroll in Brand Registry to gain access to Manage Your Experiments.
What can you test with Manage Your Experiments?
- Main product images: See what images entice customers. For example, compare minimalist photos with information-rich alternatives.
- Product titles: Find out if longer titles highlighting product features grab more attention than shorter, straightforward titles.
- Product descriptions: Test different versions of a product description to see if attributes like product dimensions compel customers to place more purchases.
- Bullet points: Do customers care more about product features, benefits, or other details highlighted in bullet points? Test alternate bullet points, varying bullet point orders, and keyword inclusion.
A+ Content
Share your brand story and enhance product listings with A+ Content, then test different modules (or different orders of modules) to see what visuals help products stand out against the competition.
Advanced features and settings
Auto-publish
If there’s a 66% chance the winning version of a title or image is better, it will automatically go live—no need to manually update the content.
Experiment to significance
Automatically conclude the experiment when there’s a 66% probability that one version of a title or image is better than the other, instead of setting a duration and waiting for results.
Start my experiment as soon as possible
The experiment will begin when the content passes validation. Disable this feature to start the experiment on a date you select.
Pro tip
Combine automated features to save time and streamline the process.
The simplified Manage Your Experiments workflow allows you to enter Version B and schedule the experiment with three pre-selected settings:
- Duration: Experiment to significance
- Publication: Auto-publish
- Start: As soon as validation is complete
If you’d rather manually adjust experiment parameters, click the View Settings toggle to see your options and update them.
Manage Your Experiments tips for success
Experiment setup
Which ASIN do you want to run the experiment on? What type of content do you want to test? How will you test it? For example, do you want to add a brand name to a product title to see if it boosts sales? Questions like these can help set a solid foundation for the experiment.
If an ASIN isn’t eligible for Manage Your Experiments, it might not have received enough traffic in recent weeks. In this case, drive more traffic to the ASIN using advertising or other means.
If an ASIN isn’t eligible for Manage Your Experiments, it might not have received enough traffic in recent weeks. In this case, drive more traffic to the ASIN using advertising or other means.
Versions A and B
Both versions of content apply to the same ASIN in a valid experiment. If you see a message that says Version A has a different ASIN, add or replace ASINs to correct mismatches.
If content doesn’t show up in the Version B drop-down, ensure that:
If content doesn’t show up in the Version B drop-down, ensure that:
- You have created a second version of content for your experiment
- The content is not in draft status
- Version B content has been submitted and approved
- Version B content is applied to the reference ASIN (the reference ASIN is the ASIN you select during experiment setup; scroll down on the Experiment Setup page to see the reference badge next to the reference ASIN)
Try to make the two versions of content as different as possible. The more different they are, the more likely it will be that any performance differences are meaningful and not caused by random chance. Manage Your Experiments notifies you when the two versions are too similar. While you can still submit the experiment, consider changing the content in these cases to increase the probability of results being statistically significant.
Duration and time frame
If you’re not sure of the best duration to choose, the Experiment to Significance feature can help by concluding the experiment when results are statistically relevant. With this setting, conclusion times will vary based on ASIN traffic.
If you want to manually set a duration, the recommended period for the experiment is 6 to10 weeks, in order to collect as much data as possible. Because of random chance, early experiment results may not be representative of the true impact of your experiment. Wait until the experiment ends to identify the winner based on high-confidence results and increase the validity of your experiment.
If you want to manually set a duration, the recommended period for the experiment is 6 to10 weeks, in order to collect as much data as possible. Because of random chance, early experiment results may not be representative of the true impact of your experiment. Wait until the experiment ends to identify the winner based on high-confidence results and increase the validity of your experiment.
Experiment launch
If content validation fails (for example, submitting an image with a non-white background in a category that requires it), Manage Your Experiments will provide an error code explaining why. When content fails validation, it will not start as scheduled. If this occurs, revise the content to resolve the issue and resubmit the experiment.
Monitoring results
Manage Your Experiments updates experiment results once a week until the experiment ends. In rare cases, results could be delayed by data processing. Processed results will automatically appear when they become available, so keep checking back.
FAQs
How can I get access to Manage Your Experiments?
Manage Your Experiments is available to brand owners registered in Brand Registry in the following Amazon stores: United States, Canada, Mexico, Germany, France, Italy, Spain, United Kingdom, Poland, Netherlands, Sweden, India, and Japan.
If you don’t see Manage Your Experiments in the navigation bar of your Seller Central account, check to make sure you’re registered as a brand owner in Brand Registry. If you registered recently, it may take several days for permissions to propagate. Grant access to secondary users from the Global Permissions page through Seller Central (login required).
If you don’t see Manage Your Experiments in the navigation bar of your Seller Central account, check to make sure you’re registered as a brand owner in Brand Registry. If you registered recently, it may take several days for permissions to propagate. Grant access to secondary users from the Global Permissions page through Seller Central (login required).
What’s the eligibility criteria for Amazon A/B tests?
An ASIN is eligible for Manage Your Experiments if it belongs to your brand and has received enough traffic in recent weeks to produce valid experiment results. Experiments on A+ Content require published A+ Content to be eligible. For more information about ASIN eligibility, see the About Manage Your Experiments page.
How do I create an effective experiment?
Plan your experiment by thinking about what type of content you want to test, and what tweaks to the content might result in higher sales. Try to make Version A and Version B very different from each other. The greater the difference, the better the chance that performance differences will be meaningful and not due to random chance. Use the Experiment to Significance setting and other advanced Manage Your Experiments features to deliver faster results.
What are the guidelines for A/B testing on Amazon?
You can only run one experiment on any particular ASIN at a time. Manage Your Experiments content for the two versions must meet standard product content guidelines. Review Amazon’s detail page and find additional information regarding content validation under Manage Your Experiments, Creating Experiments.
