How allowing Facebook to choose our audience improved donor conversion rate
Americans for Prosperity
Experiment Summary
Timeframe: 02/14/2019 - 03/14/2019
Americans for Prosperity was interested in diversifying their audience more instead of only attracting “right of center” audiences through pre-programmed target lists. We decided to run a test to see if allowing Facebook’s algorithm to choose who to run our ads to would impact our ability to convert donors.
Research Question
Will allowing Facebook to choose our audience improve donor conversion rates?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Freedom Pledge - Programmed Targeting | 0.03% | ||
| T1: | Freedom Pledge - Wide Audience Targeting | 0.05% | 66.9% | 96.8% |
This experiment has a required sample size of 75,328 in order to be valid. Since the experiment had a total sample size of 191,296, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 66.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
We were able to successfully improve donor conversion rates by 66.9% with a 96.8% level of confidence.
Other notable statistics found with this setting include:
- Revenue: +28.2% (with a 80.8% LoC)
- Email Registration Rate: +9.8% (with a 100% LoC)
- Net Cost Per Donor Acquired: -15.9%
- Recurring Donors: +501.7% (with a 91.8% LoC)
- Recurring Revenue: +10,971.9% (with a 97.9% LoC)
Both campaigns were setup with “lowest cost” setting, with a “Purchase” event, with budget optimization on the campaign level (instead of the ad set), where the only difference was pre-programming (control) audiences we wanted our ads to be displayed to versus allowing Facebook’s algorithm to target whoever it deemed most appropriate to show the ads to based upon their behavioral and interest activity.
Our campaign targeting recommendation is to always setup ads to display to (a) donors, and (b) subscribers (non-donors), but now we would recommend adding a third group — which is (c) wide audience targeting (instead of pre-programming your target audiences).