How removing Challenge Gift language affects conversion
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 12/26/2014 - 12/28/2014
As part of their year end campaign, the Heritage Foundation secured multiple gifts from major donors to serve as challenge gifts to their members. It has always been said that Challenge Gifts can increase conversion rate and average gifts but we have never known how much the increase is. We decided to test this well-known tactic to determine how much impact a Challenge Gift can have on a fundraising campaign.
Research Question
Will not including a challenge gift with the campaign have minimal impact on the fundraising campaign?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Challenge Gift Language | 0.06% | $60.68 | ||
| T1: | No Challenge Gift Language | 0.05% | -15.9% | 59.3% | $71.48 |
This experiment has a required sample size of 476,164 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
There was a slight increase in conversion rate for those that received the version with language about the challenge gift but it did not reach significance with sample size or lift. This would suggest that the challenge gift may not have the level of impact that best practices tells us it has. However, this test would need to be run with a larger segment to validate that conclusion.