How the experience on a donation confirmation page impacts subsequent gifts and overall revenue
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 02/11/2019 - 05/15/2019
A lot of focus gets put on the optimization and improvement of the actual donation process and little attention is put on the experience after a person makes a gift. We hypothesized that improving this experience and making the donor feel valued, celebrated, and affirmed in their decision to make a gift to Focus on the Family we might be able to impact and improve subsequent giving and overall revenue from the donor in the future. To do this, we developed a treatment experience on the donation confirmation page that thanked the donor by name, celebrated the donor in the copy we used, reinforced the value proposition of their gift by sharing two testimonials, and even had confetti explode on the page as soon as the donor landed on the page. We tested this approach for over six months to really assess the impact that we might be able to have on giving over a longer period of time. Half of the audience during those six months experienced the control confirmation page and the other half experienced the “Super, Happy, Thankful” treatment experience.
Research Question
Can we increase the number of subsequent gifts and overall revenue by improving the experience on the donation confirmation page?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 31.7% | ||
| T1: | Super Happy Thankful Experience | 31.6% | -0.2% | 11.0% |
This experiment has a required sample size of 3,382,646 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
To analyze the winner, we looked at donors based upon the initial treatment they saw and removed any recurring donations since they were automated. We then looked at the likelihood of a donor to make a subsequent gift, the number of subsequent gifts, and their change in revenue (did their subsequent giving increase). In all of those situations, there was no statistically significant difference between the control and treatment.
Even though we didn’t see an impact on future gifts and revenue, the treatment did a better job in thanking and celebrating the donor – which is important. Maybe the one joyful, celebratory experience on the donation confirmation page wasn’t enough to impact giving, but could we create that same experience in a series of emails every single month for the donor and have an impact on giving? We’d like to test it.