How Facebook ads impacted a campaign’s lapsed member conversion rates Experiment ID: #11967

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 2/12/2019

During The Heritage Foundation’s Kickoff Campaign, we wanted to run Facebook ads designed to remind lapsed members to renew their annual membership to see if it would boost response rates by adding a 3rd channel (digital ads) along side regularly scheduled email and direct mail messages.

Research Question

Will adding a 3rd channel of touches increase donor conversion rate for reactivating lapsed members?


C: No Ads
T1: With Ads


Treatment Name Conv. Rate Relative Difference Confidence
C: No Ads 0.10%
T1: With Ads 0.11% 7.6% 30.1%

This experiment has a required sample size of 1,406,733 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

This experiment did not validate for this donor segment, but directionally, we see alignment with the non-member and member segments that both validated towards increasing segment response rates when serving ads.

It’s worth noting that similarly to the non-member experiment we ran, the total cost to run these ads to lapsed members generated a 4.2 ROI against the increase in revenue generated from the “with ads” segment. Simply put, 100% of the ad expense to increase responses from this audience was covered and generated an amount equal to slightly more than 3x the spend in additional revenue free and clear of any expense.

Final Conclusion:
Further experimentation is required with future campaigns for the lapsed segment, but directionally these results align with active and non-member segments seeing an increase in response rates with ROI free and clear of the incremental expenses associated with adding the 3rd messaging channel of Facebook ads.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.