How adding a high-urgency sticky bar to a recurring donor campaign page affects recurring donation rates
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 03/10/2023 - 03/31/2023
During a recurring donor campaign, we wanted to know how adding a high-urgency sticky bar to the donation page would affect recurring donation rates. The sticky bar combined high-urgency language with a dynamic countdown which automatically refreshed to display how many donors were still needed to reach the campaign goal. We set this experiment up as an A/B test with traffic split roughly 50/50 between the control and the treatment.
Research Question
We believe that adding a high-urgency sticky bar on the donation page of a recurring donor campaign will achieve higher recurring donation rates by defining a tangible goal and more clearly communicating the problem being solved.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 14.4% | |||
| T1: | Sticky Bar 1 | 14.9% | 3.1% | 37.7% |
This experiment has a required sample size of 48,701 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
After running this experiment for 21 days, we observed a 5.6% increase in recurring donations produced by the treatment but with a low statistical level of confidence (36.1%). Despite the modest lift in recurring donations, we also observed that the treatment produced a sizable 21% decrease in overall revenue (with a 75.5% level of confidence), indicating that by putting more emphasis on the number of donations, the treatment may have unexpectedly had a negative impact on the amount that donors were willing to give. It may therefore be worth testing this tactic against a monetary goal in the future.