How recurring gift language impacted donor conversion
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 03/29/2018 - 06/05/2018
The Heritage Foundation was looking for ways to increase donor acquisition for their recurring giving club, The Leaders Club. As part of a series of experiments, we tested out a few different hypotheses to impact this metric. The first hypothesis we had was that simply explaining the recurring gift option and the potential benefits would increase the perceived value of the offer enough to increase acquisition for the club. We decided to test this hypothesis on the main donation form.
Research Question
Will including value proposition language specific to The Leaders Club on the main donation form increase Leaders Club Acquisition? And how will it impact overall donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Original | 9.7% | ||
| T1: | Leaders Club Language | 8.4% | -13.1% | 92.9% |
This experiment has a required sample size of 3,920 in order to be valid. Since the experiment had a total sample size of 6,652, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
After running for 2 months, we did not reach the desired level of confidence to definitively identify how this experiment impacted both the likelihood to make a gift and the likelihood to make a recurring gift. However, the impact on the likelihood to make a gift was approaching significance (93% LoC) with a 13% decrease by having the Leaders Club language on the page. The impact on the chances to make a recurring gift was a long way from statistical significance (65% LoC).
Given the likelihood that the additional value proposition was hurting conversion and that there was no definitive proof it was helping recurring gift acquisition, we decided to end the experiment and look for alternative options to increase recurring gifts.