This week, the 3Q blog takes on the topic of testing – one near and dear to all digital marketers. 

There used to be a time when split testing in Facebook required a lot of time, effort, and the nagging feeling that no matter how you split it, it still wasn’t a true split test. Thankfully those days are over with Facebook’s Split Testing capabilities, which allow you to test different variables in non-overlapping groups. Today we’re going to focus on one such variable: audiences.

Testing different audiences will help you understand what type of people are more likely to engage with your ads and convert. As a result, you also learn more about your the ideal audience.

The Basics

When you create a split test, Facebook divides your audience at the ad set level and lets you test one variable to see if one ad set (audience) performs better than the other(s). To do this, Facebook divides your audience into random, non-overlapping groups. The performance of each ad set is measured according to your campaign objective and is then recorded and compared.

Audience split testing is available for the following Business Objectives: Traffic, App Installs, Lead Generation, Conversions, Video Views, Catalog Sales, Reach, and Engagement.

How Audience Split Testing Works

Randomization is a critical element in a Facebook split test. It helps ensure the test is conducted fairly while also ensuring each ad set is given an equal chance in the auction. For the most accurate results, you’ll only have the opportunity to test one variable at a time. (If you test two audiences against each other, you can’t also simultaneously test placements because you wouldn’t know for sure which affected performance!)

Sample Audience Tests:

  • – Females living in Austin versus Females living in Chicago versus Females living in New York City
  • – A custom lookalike audience versus people who may be interested in your brand based on interest targeting

Audience Creation Tips:

  • – Your audience should be large enough to support your test
  • – Do not use your test audiences in other active campaigns; this may result in delivery problems and contaminate test results
  • – Test one specific factor for more conclusive results

Setting a Budget

Your budget will be divided between ad sets so be sure to allocate enough budget for your test. This is important! After all, what’s the point of testing if you can’t produce enough results to confidently determine a winning strategy?

If dividing equally doesn’t make sense for your needs, you can choose to weigh one more than the other(s).

Scheduling

Facebook allows between 3-14 days for testing; however, you’ll want to run and at least 4 days for the most reliable results. Tests shorter than 3 days may produce insufficient data to confidently determine a winner, and tests longer than 14 days may not be an efficient use of budget.

Results

Your winning audience will be determined by the lowest cost per result in Facebook’s attribution system.

Once testing is complete, you’ll receive a notification and email containing results. You can also view your results in Ads Manager while the test is running.

Split Testing results include:

  • – The number of times your ad achieved an outcome or result, based on the objective you selected
  • – Cost (the average cost per result from your ads)
  • – Amount spent (the total amount spent on your campaign, ad set or ad during its schedule)

Facebook will also provide a confidence level, which shows the chance you’d get the same winner if you ran this test again. If your results have low confidence, you can test the campaign again with a longer schedule or higher budget.

A percentage above 75% indicates that you should use the winning ad set. When the percentage is lower, this basically means that the ad sets performed similarly. In this case, you can use either strategy or conduct further testing.

Facebook’s guide to understanding your results depending on the number of ad sets you’re testing:


Once you understand which audience performed best, you can:

  • – Create a new ad from the winning ad set
  • – Reactivate the winning ad set in Ads Manager
  • – Create a new campaign in Ads Manager that leverages your split test learnings

For example, if you tested a Female 24-35 audience, and it performed better than a Female 36-44 audience, you could then test the 24-35 audience against a 30-35 age range to see if it continues to perform better and narrow down your demographic.

Important Things To Note

  • – Split Testing is based on people, not cookies, and gathers results across multiple devices.
  • – Changing your split test after it started may impact your test results. Use caution when editing your split test campaigns and only edit when it’s absolutely necessary.
  • – Use caution when making changes to other non-variable factors. Editing the placement, creative, and other factors can affect the results and undermine your understanding of what drove the results.
  • – If you cancel your split test before it is scheduled to end, your budget and reach will no longer be divided among ad sets, and your ads will deliver normally. No winner will be determined, and you won’t get notification or email with results.
  • – You cannot create a split test in an existing campaign. As of now, you must start with a new campaign.

 

Have you used Facebook’s Split Testing feature to find your best audiences? Tell us about your experience!

Leave a Comment

Angela Trumbaturi
Angela Trumbaturi, who joined 3Q Digital in March 2017, is a digital marketing professional with a passion for building brands and cultivating strong communities. With agency experience since 2009, she has a proven track record in strategic planning, paid social, and program execution. On weekends, you can find Angela kicking back at a festival or exploring the local food scene.