How a donation interruptor including a monthly upgrade ask affects recurring donation rates
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 01/16/2023 - 01/30/2023
In an effort to increase recurring donor acquisition for a public policy organization during a recent high-urgency campaign, we wanted to measure the effect of interrupting the donation process with a pop-up asking the donor to become a monthly supporter instead of making only a one-time gift.
This pop-up was set up as an A/B test that would only fire for 50% of donors who were in the process of completing a one-time gift under $100, reiterating the value of becoming a monthly donor, both in terms of impact and exclusive benefits, and asking them to a) make a recurring monthly donation of $20 or b) proceed with their one-time gift.
Research Question
We believe that
presenting a monthly upgrade ask for one-time donors during the donation process will achieve higher recurring donation rates by increasing its visibility at the decision-making point and prominently communicating its value.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.56% | |||
| T1: | Donation interrupter | 1.1% | 93.6% | 99.3% |
This experiment has a required sample size of 2,296 in order to be valid. Since the experiment had a total sample size of 8,599, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 93.6% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running this experiment for approximately 2 weeks, we observed a 93% increase in monthly giving with a 99.2% level of confidence.
We believe that several motivating factors came into play in producing this result:
- First, the interrupter pop-up dramatically increased the visibility of the recurring-giving option.
- Second, it clearly and thoroughly laid out the value of becoming a recurring donor in terms of impact and exclusive benefits
- Third, by creating a mental interruption, it may have given donors an opportunity to reset the mental process of completing their donation (transaction forms being as prevalent as they are in the digital space may cue a certain degree of auto-pilot during the transaction process) and allowed them the time to consider their options.
- And finally, the pop-up treatment positioned monthly giving as an opportunity rather than an ask, giving agency to the donor and casting them in the role of someone who is able to do (and get) more than they would by making a one-time gift.
And while at first glance it appears that the treatment produced a 12.6% decrease in donations and a 16.4% decrease in revenue, upon further examination it appears unlikely that these dips were caused by the pop-up since viewers technically “enter” the experiment upon page-load and the pop-up does not appear until they have already made the decision to give. Additionally, language in the headline of the pop-up indicates to the donor that their donation has not yet been processed to reduce instances of closing the browser window before the donation process is complete.
This leads us to believe that any decrease in donations and/or revenue is more likely a result of the visitor deciding to give when viewing the page, rather than a result of the pop-up itself.
Based on these results, we will explore additional opportunities to globalize this learning.