How recurring gift language impacted donor conversion Experiment ID: #9094

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 3/29/2018 - 6/5/2018

The Heritage Foundation was looking for ways to increase donor acquisition for their recurring giving club, The Leaders Club. As part of a series of experiments, we tested out a few different hypotheses to impact this metric. The first hypothesis we had was that simply explaining the recurring gift option and the potential benefits would increase the perceived value of the offer enough to increase acquisition for the club. We decided to test this hypothesis on the main donation form.

Research Question

Will including value proposition language specific to The Leaders Club on the main donation form increase Leaders Club Acquisition? And how will it impact overall donor conversion?

Design

C: Original
T1: Leaders Club Language

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Original 9.7%
T1: Leaders Club Language 8.4% -13.1% 92.9%

This experiment has a required sample size of 3,920 in order to be valid. Since the experiment had a total sample size of 6,652, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

After running for 2 months, we did not reach the desired level of confidence to definitively identify how this experiment impacted both the likelihood to make a gift and the likelihood to make a recurring gift. However, the impact on the likelihood to make a gift was approaching significance (93% LoC) with a 13% decrease by having the Leaders Club language on the page. The impact on the chances to make a recurring gift was a long way from statistical significance (65% LoC).

Given the likelihood that the additional value proposition was hurting conversion and that there was no definitive proof it was helping recurring gift acquisition, we decided to end the experiment and look for alternative options to increase recurring gifts.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.