How Facebook ads impacted a campaign’s lapsed member conversion rates

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 02/12/2019

During The Heritage Foundation’s Kickoff Campaign, we wanted to run Facebook ads designed to remind lapsed members to renew their annual membership to see if it would boost response rates by adding a 3rd channel (digital ads) along side regularly scheduled email and direct mail messages.

Research Question

Will adding a 3rd channel of touches increase donor conversion rate for reactivating lapsed members?

Design

C: No Ads

T1: With Ads

Results

This experiment has a required sample size of 1,406,733 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

This experiment did not validate for this donor segment, but directionally, we see alignment with the non-member and member segments that both validated towards increasing segment response rates when serving ads.

It’s worth noting that similarly to the non-member experiment we ran, the total cost to run these ads to lapsed members generated a 4.2 ROI against the increase in revenue generated from the “with ads” segment. Simply put, 100% of the ad expense to increase responses from this audience was covered and generated an amount equal to slightly more than 3x the spend in additional revenue free and clear of any expense.

Final Conclusion:
Further experimentation is required with future campaigns for the lapsed segment, but directionally these results align with active and non-member segments seeing an increase in response rates with ROI free and clear of the incremental expenses associated with adding the 3rd messaging channel of Facebook ads.