How optimizing for Subscriptions on Facebook impacts recurring donor acquisition
Care Net
Experiment Summary
Timeframe: 05/18/2020 - 06/20/2020
Roughly a year ago, Care Net ran an experiment to determine the impact of using different optimization methodologies within Facebook. In that experiment, we learned that optimizing our Facebook campaigns to acquire “Purchasers” yielded more donors than optimizing for ”Custom Conversions” (aka emails) or even for optimizing for “Donors” as Facebook defines them. Our hypothesis for this result is that Facebook’s advertising algorithm incorporated online buying behavior with ”Purchase” optimization that the other metrics did not. Ultimately, this experiment drastically improved acquisition efforts and significantly lowered costs.
Recently, Facebook rolled out additional conversion options that are available for optimization. One of those options is a ”Subscription” which is intended to represent an ongoing monthly payment. Given Care Net’s emphasis on acquiring recurring donors, we decided to run an experiment to see if this metric could acquire more recurring donors than “Purchase” optimization.
Research Question
Will optimizing Facebook ads for “Subscriptions” increase recurring donors when compared to “Purchases?”
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Purchase Optimization | 0.05% | ||
| T1: | Subscription Optimization | 0.03% | -43.4% | 97.0% |
This experiment has a required sample size of 65,145 in order to be valid. Since the experiment had a total sample size of 173,034, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 43.4% decrease in conversion rate
× 0% increase in average gift
Key Learnings
After running for a full month, we found that by optimizing for Subscriptions (AKA, recurring gifts), we actually decreased the number of recurring gifts that we received by 43%. Our hypothesis is that when Facebook optimizes for Purchases, it incorporates online spending into its algorithm which increases the likelihood that someone the ad targets would make a financial transaction. With Subscriptions, it would appear that this same financial information in not incorporated into the results.
Additionally, this experiment had a negative impact on several other conversion factors
- Decreased total donors by 50.9% with a 99.8% LoC
- Decreased the annual the number of gifts (assuming 12 gifts per recurring gift) by 42% with a 99.9% LoC
It is important to note that the media spend for these two campaigns were nearly identical. Seeing the difference in the number of people reached illustrates that it costs more to optimize for Purchases but that it gets better “quality” audiences in terms of donor conversion.
The next iteration of this experiment would be segmenting the Purchase event by the type of gift which is something that can be passed through the meta information with the donation.