How allowing Facebook to choose our audience improved donor conversion rate Experiment ID: #12564

Americans for Prosperity

Experiment Summary

Timeframe: 2/14/2019 - 3/14/2019

Americans for Prosperity was interested in diversifying their audience more instead of only attracting “right of center” audiences through pre-programmed target lists. We decided to run a test to see if allowing Facebook’s algorithm to choose who to run our ads to would impact our ability to convert donors.

Research Question

Will allowing Facebook to choose our audience improve donor conversion rates?

Design

C: Freedom Pledge - Programmed Targeting
T1: Freedom Pledge - Wide Audience Targeting

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Freedom Pledge - Programmed Targeting 0.03%
T1: Freedom Pledge - Wide Audience Targeting 0.05% 66.9% 96.8%

This experiment has a required sample size of 75,328 in order to be valid. Since the experiment had a total sample size of 191,296, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 66.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

We were able to successfully improve donor conversion rates by 66.9% with a 96.8% level of confidence.

Other notable statistics found with this setting include:

  • Revenue: +28.2% (with a 80.8% LoC)
  • Email Registration Rate: +9.8% (with a 100% LoC)
  • Net Cost Per Donor Acquired: -15.9%
  • Recurring Donors: +501.7% (with a 91.8% LoC)
  • Recurring Revenue: +10,971.9% (with a 97.9% LoC)

Both campaigns were setup with “lowest cost” setting, with a “Purchase” event, with budget optimization on the campaign level (instead of the ad set), where the only difference was pre-programming (control) audiences we wanted our ads to be displayed to versus allowing Facebook’s algorithm to target whoever it deemed most appropriate to show the ads to based upon their behavioral and interest activity.

Our campaign targeting recommendation is to always setup ads to display to (a) donors, and (b) subscribers (non-donors), but now we would recommend adding a third group — which is (c) wide audience targeting (instead of pre-programming your target audiences).


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.