How Facebook ads impacted a campaign’s non-member conversion rates Experiment ID: #11966

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 2/12/2019

During The Heritage Foundation’s Kickoff Campaign, we wanted to run Facebook ads designed to remind members to activate their membership to see if it would boost response rates by adding a 3rd channel (digital ads) along side regularly scheduled email and direct mail messages.

Research Question

Will adding a 3rd channel of touches increase donor conversion rate for activating non-members for the first time?


C: No Ads
T1: With Ads


Treatment Name Conv. Rate Relative Difference Confidence
C: No Ads 0.08%
T1: With Ads 0.10% 20.3% 93.5%

This experiment has a required sample size of 243,412 in order to be valid. Since the experiment had a total sample size of 433,050, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

With a 99.9% level of confidence, the non-member segment that received Facebook ads in the 3rd channel of communication saw an online response rate increase of 20.3%.

Other metrics impacted are as follows:

  • Average gift: +13.1% increase
  • Total Revenue: +36.09% increase

During the campaign, the total cost to run these ads to non-members generated a 2.03 ROI against the increase in revenue generated from the “with ads” segment. Simply put, 100% of the ad expense to increase responses from this audience was covered and generated an equal amount to the budget spent in additional revenue free and clear of any expense.

Final Conclusion:
Where possible, adding a 3rd channel of communication to heed the campaign’s call to action for non-members should be considered.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.