How a two-step donation form impacts donor conversion on mobile devices during year-end
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 12/14/2020 - 12/26/2020
During Focus on the Family’s year-end campaign, we wanted to see if we could improve donor conversion on mobile devices. Mobile donor conversion has been something that has been difficult to impact in the past during high-urgency campaigns. So to continue to test into this, we developed a more radical giving experience by splitting the donation form into two steps. We hypothesized that it might be easier and a better user experience to break the form into separate steps rather than showing people the full form all at once. We hypothesized that we might be able to decrease the perceived cost of the number of form fields on the page with the two-step approach.
Research Question
Will a two-step donation form increase donor conversion on mobile devices during a high-urgency campaign?
The two-step form saw a directional lift of 22%! While we weren’t able to validate the results, we were able to see a positive increase in conversion rate! This positive directional lift is mostly due to the high volume of email traffic coming to the donation form. People’s motivation is slightly higher from this traffic source and the decrease in perceived friction to the giving experience helped to convert more of these people. We would like to test this again during non high-urgency campaigns. We would also like to test into the order of the information we are asking for from people. Take a look at other interesting data points from specific segments:
- New visitors had a 19.4% increase in donations with an 87% level of confidence.
- Email visitors had a 42.3% increase in donations with a 97% level of confidence.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 5.1% | ||
| T1: | Two Step Form | 6.3% | 21.9% | 89.7% |
This experiment has a required sample size of 3,271 in order to be valid. Since the experiment had a total sample size of 4,529, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
The two-step form saw a directional lift of 22%! While we weren’t able to validate the results, we were able to see a positive increase in conversion rate! This positive directional lift is mostly due to the high volume of email traffic coming to the donation form. People’s motivation is slightly higher from this traffic source and the decrease in perceived friction to the giving experience helped to convert more of these people. We would like to test this again during non high-urgency campaigns. We would also like to test into the order of the information we are asking for from people. Take a look at other interesting data points from specific segments:
- New visitors had a 19.4% increase in donations with an 87% level of confidence.
- Email visitors had a 42.3% increase in donations with a 97% level of confidence.