Create an A/B test task | SAS

SAS® Customer Intelligence 360



Create an A/B test task

Learn how to create an A/B test task to compare the performance of multiple creatives.


To set up an A/B test task:
  1. From the side menu, click Tasks.
  2. Click Create new task and select the task type.
    – Select Mobile A/B Test to compare creatives in a mobile app.
    – Select Web A/B Test to compare creatives on a website.
    When you create a task, it has a status of Designing. A Designing status indicates that the task is still being worked on and is not ready to be published. 
  3. Select the spot where you want your variants to be displayed. Click Create Task
  4. The Content tab contains the details that are associated with the content that the task delivers. Click Select Message to start adding variants and setting their properties. In the properties pane, you can complete these actions: 

    1. Set the distribution of the variant frequency. 
    2. Optimize the test so that the system can change the distribution of variants as the A/B test progresses. Optimizing the test ensures that the best-performing variant gets a larger portion of the distribution. 
    3. Add or remove variants. After the task is published, variants cannot be added or removed. 
    4. Rename variants. 
    5. Change the champion. After the task is published, the champion cannot be changed. 
    6. Set impression limits. 
    7. Change the creative for a variant. After the task is published, a variant’s creative cannot be changed.

      Note: When you add a message with multiple creatives to an A/B test task, you have the option to automatically set each creative as a variant. You can also add variants while you are designing your task by navigating to Content > Variants.
  5. On the Targeting tab, define the segment that sees the content that is delivered by the task. Add criteria to the Eligible page to specify which people to include in your segment. Add criteria to the Exclusions page to specify which people to exclude from your segment.
  6. On the Goals tab, add the goals for the task. After the task is published, goal events cannot be changed. 
  7. On the Orchestration tab, set the priority of your task compared to other tasks that deliver content to the same spot. Choose 1 (Highest) to give your task priority over other tasks. Choose 5 (Lowest) to give other tasks priority over your task.
    Note: Although you cannot set a priority for an A/B test task, it always takes priority over a task that is not an A/B test task. For example, you have a Web task and a Web A/B test task that deliver content to the same spot. The Web A/B test task always delivers its content, no matter what the priority is of the Web task.
  8. If you are finished updating your task, you can set when you want to publish the task. A task does not deliver content until it is published and has a status of Active. On the Orchestration tab, complete one of these actions:

    1. Click Mark Ready when you are finished updating the task but do not want to publish the task. This action changes the status of the task from Designing to Ready. Click Return to Designing if you need to revert to the previous status.
    2. Click Mark Ready and Publish or Publish without setting a schedule if you want the task to deliver content immediately. This action changes the status of the task to Active. The task cannot be changed back to Ready or Designing.
    3. On the Schedule page, click Schedule task and set a start date and an end date for the task. Then click Mark Ready and Publish or Publish. This action changes the status of the task to Scheduled. When the start date is reached, the creatives are delivered to the spot and the task’s status changes to Active. The task cannot be changed back to Ready or Designing.

      Note: If your task has a status of Ready and a start date, the task is not automatically published once the start date is reached. A task must be published in order for the start date to activate the task. Changes that are made to a scheduled or active task are not automatically propagated to the published version of the task. If you update an active or scheduled task, republish the task after you save it to resolve the Out of Sync status. If the task is not republished, it stays out of sync with the published version and the updates are not delivered.
  9. While your task is running, it collects data and gathers information about progress toward your goals. For details about how the variants are performing, navigate to the Goals tab. The Goals tab contains the measurements for each variant. When a confidence level of 75% for the lift value is reached, you can also see the recommended variant winner and any discovered segments. To see general information about the task, including some of the information that is available on the Goals tab, navigate to the Insights and Summary tabs.
  10. When you no longer need the task to run or you are ready to declare a winning variant, end the task. There are three ways to end an A/B task:

    1. Click End on the Orchestration tab.
    2. The end date that is specified in Orchestration > Schedule > Orchestration arrives.
    3. On the Goals tab, click Declare Winner and select a winning variant. When you declare a winner, a new task is created of the same type in order to ensure that content continues to be delivered to the spot.

      Ending a task changes the status of the task from Active to Complete. Completed tasks do not collect new data and cannot be modified. You can navigate to the Goals tab of a completed task to review the existing measurements and data. In addition, tasks that are Complete do not deliver content to a spot. Any unsaved changes to the task are lost.

Note: If you want to create a stand-alone task from an open A/B test task that contains the same targeting, spot, and goals, open the A/B test task and click Options. Select Create new task, and then select the variant that you want to use in your new task.

Back to Top