The Heritage Foundation

How urgency impacted donor conversion on The Daily Signal donation ad

Experiment ID: #2807

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 12/21/2016 - 01/01/2017

The Heritage Foundation had recently discovered a tactic that was able to successfully increase the donor conversion rate of readers of The Daily Signal. This website is a Heritage-owned property that receives over 2 million visitors per month so even minor improvements can lead to significant gains.

In the days leading up to December 31st, we wanted to see if we could utilize the upcoming deadline to increase the urgency of the gift. For this test, we modified the existing value proposition statement to allude to the December 31st deadline for tax-deductibility and then added a countdown clock to visually reinforce this date.

We then launched the experiment with 10 days remaining in the year.

Research Question

Will including language and a countdown clock about the approaching year end increase donor conversion rate?


C: Journalism Value Prop
T1: Year End Message


 Treatment NameConv. RateRelative DifferenceConfidence
C: Journalism Value Prop 0.08%
T1: Year End Message 0.12%61.1% 100.0%

This experiment has a required sample size of 35,777 in order to be valid. Since the experiment had a total sample size of 418,364, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 61.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

This experiment had two interesting observations. The first, and most important, is that we were able to lift donor conversion rate by 61% by reinforcing the urgency of the December 31st deadline. The second observation came from an analysis of the number of visitors that clicked on the initial ad.

What we found is the year end messaging resulted in a 45.6% decrease in click through rate (statistically significant). That means we were able to achieve a 61% lift in donations from roughly half the number of visitors. The key to this learning is that, while the urgency-based message may not be as appealing to a larger audience, it increases the motivation of those that do click on the link.

Experiment Documented by NextAfter

Question about experiment #2807

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.