How simplifying a donation form affects revenue
Jews for Jesus
Jews for Jesus exists to make the messiahship of Jesus an unavoidable issue to the Jewish people worldwide, utilizing creative evangelistic methods to reach the Jewish community and spread the good news of Jesus Christ.
Experiment Summary
Timeframe: 06/17/2015 - 06/19/2015
Like many organizations, Jews for Jesus uses a content management system to quickly produce donation form variations for each monthly appeal. The style and layout of the default form is something that is the result of multiple modifications and functionality improvements over the years. When this is the situation, sometimes it helps to take a step back and re-evaluate what has become the standard.
Jews for Jesus decided to do a radical redesign of the standard donation form. We wanted to reduce the number of decisions in the giving process and make the interface easier to use for visitors. The radio buttons on the control were very tiny and difficult to use, especially on mobile devices. They were replaced in the treatment with large clickable buttons. The option to log in was also removed. While this is intended to provide ease of use for repeat donors, it also adds friction, anxiety, and another point to exit the giving process. Finally, we removed the option to leave a comment. While comments are certainly welcome in the donation process, they may increase friction if they come before the actual donation takes place. We then launched this new treatment as part of one of the appeals in a campaign.
Research Question
Which donation form will have the biggest impact on revenue?
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Standard Donation Form | $1.46 | ||
| T1: | Simplified Form | $5.15 | 252.6% | 99.7% |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 63.1% increase in conversion rate
× 116.2% increase in average gift
252.6% increase in revenue
Key Learnings
The simplified form resulted in a 63.1% increase in conversion rate and a 116% increase in average gift. When factored together, this equates to a 252.6% increase in overall revenue.
The simplified form does not contain all of the functionality of the previous form, such as the ability to retrieve a saved a credit card or to opt into the monthly newsletter. However, by removing some of this functionality, we reduced the number of decisions donors had to make (lowering friction) and improved the overall results of the form.