Learn how to compare creatives by testing them to find the one that performs best.
A variant is a creative that you are testing. A minimum of two variants is required before you can publish an A/B test task, and one of the variants must be denoted as the champion variant. An A/B test task cannot contain more than 20 variants. A champion variant is the control variant in the A/B test task. The performance of other variants that are included in the task is compared against the performance of the champion in order to determine the overall variant winner. For example, you might have one creative that is currently in use on your website and three new creatives that have not been delivered yet. You can create a web A/B test task that compares the current creative and the new creatives. In the A/B test task, you can add all of the creatives and label the current creative as the champion variant.
In your A/B test task, an option allows the system to optimize the distribution of the variants while the test is running. This option allows the system to change the distribution of variants as the A/B test progresses so that the best-performing variant gets a larger portion of the distribution. The variant distribution changes only when there is enough data to identify the best-performing variants. The analytics that optimize the task are run approximately every 12 hours.
Note: Optimizing the distribution of the variants prevents new segments from being discovered by the analytics.
Find the winning creative
The variant that performs the best against the target audience is denoted as the overall variant winner. Before the overall variant winner can be determined, a minimum sample size of how many people need to access the content must be met.
After a minimum confidence level is reached, the system identifies the overall variant winner. You can then decide to end the test task and create a new task with the winning creative.
As the A/B test progresses, the system’s analytics might discover that one variant out-performs the overall variant winner for a subset of your target audience. This subset of people is called a discovered segment, and you can access the profile for a discovered segment on the Goals tab. The profile includes details about the discovered segment, including which criteria distinguish the discovered segment from the segment defined in targeting.
Note: Discovered segments are not supported in A/B test tasks that optimize the distribution of the variants.
Create a standalone task from an A/B test task
You can create a standalone task that contains the same targeting, spot, and goals as an A/B test task. Open the A/B test task and click Options. Select Create new task, and then select the variant that you want to use in your new task. You can also create a standalone task from the winning variant.