How urgency impacted donor conversion on The Daily Signal donation ad
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 12/21/2016 - 01/01/2017
The Heritage Foundation had recently discovered a tactic that was able to successfully increase the donor conversion rate of readers of The Daily Signal. This website is a Heritage-owned property that receives over 2 million visitors per month so even minor improvements can lead to significant gains.
In the days leading up to December 31st, we wanted to see if we could utilize the upcoming deadline to increase the urgency of the gift. For this test, we modified the existing value proposition statement to allude to the December 31st deadline for tax-deductibility and then added a countdown clock to visually reinforce this date.
We then launched the experiment with 10 days remaining in the year.
Research Question
Will including language and a countdown clock about the approaching year end increase donor conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Journalism Value Prop | 0.08% | $61.48 | ||
| T1: | Year End Message | 0.12% | 61.1% | 100.0% | $99.64 |
This experiment has a required sample size of 35,777 in order to be valid. Since the experiment had a total sample size of 418,364, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 61.1% increase in conversion rate
× 62.1% increase in average gift
161.1% increase in revenue
Key Learnings
This experiment had two interesting observations. The first, and most important, is that we were able to lift donor conversion rate by 61% by reinforcing the urgency of the December 31st deadline. The second observation came from an analysis of the number of visitors that clicked on the initial ad.
What we found is the year end messaging resulted in a 45.6% decrease in click through rate (statistically significant). That means we were able to achieve a 61% lift in donations from roughly half the number of visitors. The key to this learning is that, while the urgency-based message may not be as appealing to a larger audience, it increases the motivation of those that do click on the link.