How simplifying a donation form affects revenue Experiment ID: #1434

Jews for Jesus

Jews for Jesus exists to make the messiahship of Jesus an unavoidable issue to the Jewish people worldwide, utilizing creative evangelistic methods to reach the Jewish community and spread the good news of Jesus Christ.

Experiment Summary

Timeframe: 6/17/2015 - 6/19/2015

Like many organizations, Jews for Jesus uses a content management system to quickly produce donation form variations for each monthly appeal. The style and layout of the default form is something that is the result of multiple modifications and functionality improvements over the years. When this is the situation, sometimes it helps to take a step back and re-evaluate what has become the standard.

Jews for Jesus decided to do a radical redesign of the standard donation form.  We wanted to reduce the number of decisions in the giving process and make the interface easier to use for visitors. The radio buttons on the control were very tiny and difficult to use, especially on mobile devices. They were replaced in the treatment with large clickable buttons. The option to log in was also removed. While this is intended to provide ease of use for repeat donors, it also adds friction, anxiety, and another point to exit the giving process. Finally, we removed the option to leave a comment. While comments are certainly welcome in the donation process, they may increase friction if they come before the actual donation takes place. We then launched this new treatment as part of one of the appeals in a campaign.

Research Question

Which donation form will have the biggest impact on revenue?

Design

C: Standard Donation Form
T1: Simplified Form

Results

Treatment Name Revenue per Visitor Relative Difference Confidence
C: Standard Donation Form $1.46
T1: Simplified Form $5.15 252.6% 99.7%

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 63.1% increase in conversion rate
× 116.2% increase in average gift
252.6% increase in revenue

Key Learnings

The simplified form resulted in a 63.1% increase in conversion rate and a 116% increase in average gift.  When factored together, this equates to a 252.6% increase in overall revenue.

The simplified form does not contain all of the functionality of the previous form, such as the ability to retrieve a saved a credit card or to opt into the monthly newsletter.  However, by removing some of this functionality, we reduced the number of decisions donors had to make (lowering friction) and improved the overall results of the form.


Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.


Pick your format of choice:

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.