How simplifying the primary header on the Focus site for mobile devices impacts donor conversion
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 08/25/2021 - 09/10/2021
Focus on the Family was running a high urgency giving campaign called SeeLife. During the campaign they featured a donation wikibanner at the top of the site’s primary header and navigation. When looking at the experience on mobile devices, there was a lot of clutter with the addition of this wikibanner to what was already on the site. We hypothesized this was causing friction to the donation pathway. To test into this, we developed two variations. For one of them, we removed the CTA’s in the header that were not needed to make a gift, but kept the “donate” button. For the second version, we removed the CTA’s in the header that were not needed to make a gift, removed the donate button, but added a line of copy above the gift array that said, “please select the amount you would like to give”.
Research Question
Can we increase donor conversion by removing additional clutter and creating clarity to the navigation header?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.27% | |||
| T1: | Keep the Donate button | 0.26% | -1.8% | 18.7% | |
| T2: | Gift Arrays Copy | 0.30% | 11.3% | 84.1% |
This experiment has a required sample size of 256,822 in order to be valid. Since the experiment had a total sample size of 366,607, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
While we weren’t able to validate the experiment, but we did see get a directional lift with one variation and got even greater clarity with the user experience.
We are learning that donor’s are experiencing the most friction with the wikibanner itself. When we added the call to action on top of the gift array, we were able to increase donor conversion (directionally) by 11%! Revenue experienced a 15.4% increase with a 92% level of confidence.
Just adding the donate button back in to the experience had no impact on donor conversion.
Observations of the impact on other visitor segments in the experiment:
- Paid Search visitors had a 113.8% increase in donations with a 99% level of confidence.
- New visitors had a 12.5% increase in donations with an 89% level of confidence.
- 35-44 visitors had a 127.1% increase in donations with a 100% level of confidence.
- 45-54 visitors had an 84.4% increase in donations with a 98% level of confidence.