How the order of a donation ask impact donor conversion
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 07/14/2017 - 09/05/2017
In a previous test, we had found that an inline donation ask, was able to increase the number of donations that were received from readers of Heritage.org articles. However, despite the percentage increase, the total number of donations was still not very high.
We had the hypothesis that this audience might not yet be ready for a donation ask. Instead, we wanted to see we could get significantly more email addresses that would outweigh the immediate value received from the donations. We would set up the email acquisition offer to also have an instant donation ask for immediate conversion and we also know that we can convert 1% to 5% of all emails acquired downstream. Based upon these factors, we wanted to see if offering an email acquisition offer before the donation ask would achieve better long-term results.
Research Question
What type of offer will lead to the greatest return in terms of email subscribers and donations acquired?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Donation Ask | 0.01% | ||
| T1: | Email Ask | 0.00% | -9.6% | 25.1% |
This experiment has a required sample size of 15,104,910 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
We found that there was no statistically significant difference in donations between the two treatments. However, during this same period, the email offer also collected nearly 1,000 new subscribers for the house email file. Given the long term conversion that is likely to happen with these new subscribers, this makes the email acquisition offer a clear winner.