How different audience motivations affect click rate in a Facebook ad (Part 1)
NextAfter
Experiment Summary
Timeframe: 07/28/2016 - 08/01/2016
While setting up the advertising and promotions for an upcoming live broadcast, we wanted to determine if our various audience groups for our Facebook Ads would respond differently to various ad treatments. We have 3 main audience groups that we ran this test with: our housefile, housefile lookalikes, and nonprofit professionals.
Our original live broadcast Facebook Ad focused on the speaker and the title, but we thought people may be more motivated by different aspects of the focus topic. For the treatments, we developed an ad that focused on “Growing Your Donor File” and one that focused on “Empathy.”
Research Question
Which ad variation will the nonprofit professionals audience respond to more?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Speaker | 0.30% | ||
| T1: | Donor File | 1.1% | 277.1% | 96.2% |
| T2: | Empathy | 0.40% | 33.6% | 22.0% |
This experiment has a required sample size of 1,309 in order to be valid. Since the experiment had a total sample size of 2,937, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
277.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The “Growing Your Donor File” treatment greatly out performed both the control and treatment 2, lifting click rate by 277% with the nonprofit professionals audience. This learning will allow us to increase our total leads for this specific live broadcast. For future live broadcasts, it shows us that it will be more effective to focus our advertising on the learning and results that someone will get out of it, rather than the speaker and their qualifications.