How a custom Facebook conversion event to target recurring purchases impacted recurring donor conversion rate
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 08/26/2020 - 09/25/2020
As a part of experimenting with Facebook’s custom conversion events, we wanted to experiment with whether the custom-defined conversion event which specifically identified and send payment details back to Facebook’s algorithm to differentiate between a single gift and a monthly recurring gift purchase type would impact the recurring donor conversion rate.
For this campaign, we partnered with Wiland to identify a custom audience of lookalike donors who are most likely (per the Wiland model) to become a recurring donor for The Heritage Foundation.
The only variable in this experiment was (a) the standard, out of the box, “purchase” event from Facebook vs. (b) the custom-defined “recurring purchase” event that we created and installed in Facebook.
Research Question
Would a custom “recurring purchase” event type increase a campaign’s recurring donor conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Standard "Purchase" Event | 0.02% | ||
| T1: | Custom "Recurring Purchase" Event | 0.01% | -63.9% | 99.5% |
This experiment has a required sample size of 55,328 in order to be valid. Since the experiment had a total sample size of 252,098, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 63.9% decrease in conversion rate
× 0% increase in average gift
Key Learnings
With a 99.5% level of confidence, we observed a significant decrease of -63.9% in recurring donor conversion rate. We observed a -62% decrease in revenue (essentially, a proportionate decrease relative to the lesser quantity of donors coming through the treatment experience) with a 100% level of confidence, as well.
It should be noted that we turned this experiment off only about 72% of the way through the minimum number of transactions we would like to see in a donor conversion study of this type (and even less—about 35%—of a revenue lift study), but the volume was still high enough and the spread great enough between the control and treatment experiences that we believed it would be best to not finish the study with the target minimum number of gifts we would normally like to see.
Further experimentation may be required with the purchase event programming and audience targeting combinations, but this certainly seems to point towards a decrease in donor conversion for recurring donor-focused campaigns.