The Heritage Foundation

Will a year-end specific message with a countdown clock in the “Dear Reader” ads drive increased donor conversions from readers of The Daily Signal?

Experiment ID: #9942

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 12/17/2018 - 12/23/2018

The Daily Signal’s inline donation ask shown at the bottom of every article is a significant driver of revenue for them. During the year end campaign, we wanted to see if a targeted message that aligned to the overall campaign would increase giving. We included an animated countdown clock and the December 31 deadline within the copy to help increase the perceived urgency of the appeal.

Research Question

Will emphasizing the year end deadline increase donations in the weeks prior to the last week of December?

Design

C: Membership
T1: Dec 31 Deadline

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Membership 0.07%
T1: Dec 31 Deadline 0.18%176.7% 100.0%

This experiment has a required sample size of 6,833 in order to be valid. Since the experiment had a total sample size of 46,320, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 176.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

The extremely relevant year-end message resonated with readers, while the countdown clock built a sense of urgency in them, as well. These two components combined to drive a lift of 176.7% increased donor conversion rate

When looking at post-click activity, we also noted that the conversion rate reached a staggering 37.6% for visitors that arrived to the donation page. The details are listed below.

Control:

  • Visitors: 132
  • Conversions: 16
  • Onsite Conversion Rate: 12.1%

Treatment:

  • Visitors: 109
  • Conversions: 41
  • Onsite Conversion Rate: 37.6%

Finally, we also noted that the average gift was 12% higher for the treatment, even though the control generated a 10% better clickthrough rate when compared to the treatment “Dear Reader” ad.

Because we had limited time before the calendar year-end fundraising campaign messaging could be used, we decided to roll the treatment out as the new control to take maximize giving opportunities during the campaign.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #9942

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.