How the addition of a countdown clock on a pop-up impacts donors conversion during year-end.
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 12/27/2017 - 01/01/2018
To our surprise, the treatment version with the countdown clock on the pop-up actually decreased donations by 12%.
Research Question
Would a countdown clock on the pop-up increase donations?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Original | 1.4% | ||
T1: | Countdown Clock | 1.1% | -18.2% | 100.0% |
This experiment has a required sample size of 15,145 in order to be valid. Since the experiment had a total sample size of 224,690, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 18.2% decrease in conversion rate
× 0% increase in average gift
Key Learnings
In the final days of Focus on the Family’s year-end giving campaign, they wanted to make it easy for donors to give. To do this, they had a pop-up appear when you visited their website. We hypothesized that we might be able to increase motivation by creating a greater sense of urgency on this pop-up with the addition of a countdown clock. The countdown clock showed people how much time they had left to give until the end of the year. We know that urgency is a motivating factor during online giving campaigns like year-end. We split the traffic that would see each version of the pop-up and tested it.
Question about experiment #2456
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.