How a long-form, donor-centered appeal decreased donor conversion rates on the primary donate page
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 03/08/2019 - 03/13/2019
We wanted to test a donor-centered message on the primary donation page, as every treatment historically has been focused on an organizationally centered series of value propositions.
Research Question
Does a donor centered message help improve donor conversion rate (or revenue) on the primary donation form for Heritage?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control - Original | 15.7% | ||
| T1: | Treatment - Donor Centered Message | 9.7% | -37.8% | 94.8% |
This experiment has a required sample size of 242 in order to be valid. Since the experiment had a total sample size of 472, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
With a 94.7% level of confidence, we validated that the treatment decreased the donor conversion rate by -37.8%.
Although the donor conversion rate decreased in the treatment segment, because the average gift in the treatment jumped pretty substantially (+61.3%) compared to the control, the revenue difference between the segments was only -0.6% (with a 50.86% LoC).
Our hypothesis is that the copy length led to the decrease in donor conversion rate, but we were encouraged by the increase in average gift from the treatment, so we will now develop and launch a short-form version of a donor-centered message to experiment against the short-form control message.