How a tabbed donation form impacts recurring donor acquisition
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 12/13/2019 - 01/09/2020
During year end, we wanted to find ways to increase membership to the Leaders Club for The Heritage Foundation. One tactic that had worked with a similar organization was utilizing a tabbed donation form. This allowed us to adjust the suggested gift array lower for recurring donors and changed the order of the mental conversation to ask whether they wanted to give recurring prior to asking how much they wanted to give.
Research Question
We believe that using a tabbed form for visitors to /renew and the main donation page will achieve increased recurring donor acquisition.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.80% | $0.00 | ||
| T1: | Tabbed Recurring Gifts | 9.5% | 1,090.5% | 100.0% | $0.00 |
This experiment has a required sample size of 50 in order to be valid. Since the experiment had a total sample size of 3,788, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 1,090.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
We observed a 1,029% increase to recurring donor acquisition with the treatment. However, after a deeper analysis, we discovered that many of these donor didn’t realize they had been making a recurring gift. This is due to the recurring option being pre-selected for all visitors.
Since we had no way of knowing what gift type they were trying to give, we decided to cancel all recurring gifts and rerun the experiment in the new year without selecting the recurring option for them.
Notable changes to other metrics due to the experiment:
- Donations experienced a 22.2% decrease with a 100% level of confidence.
- Revenue experienced a 35.9% decrease with a 98% level of confidence.
Observations of the impact on other visitor segments in the experiment:
- Desktop visitors had a 1,662.5% increase in monthly recurring gift with a 100% level of confidence.
- Mobile visitors had a 1,609.8% increase in monthly recurring gift with a 100% level of confidence.
- New visitors had an 8,762.1% increase in monthly recurring gift with a 100% level of confidence.
- Returning visitors had a 572.5% increase in monthly recurring gift with a 100% level of confidence.
- Organic Search visitors had a 10.6% increase in monthly recurring gift with a 99% level of confidence.