Jews for Jesus

How emphasizing a primary call to action in the navigation can affect donations

Experiment ID: #2092

Jews for Jesus

Jews for Jesus exists to make the messiahship of Jesus an unavoidable issue to the Jewish people worldwide, utilizing creative evangelistic methods to reach the Jewish community and spread the good news of Jesus Christ.

Experiment Summary

Timeframe: 09/08/2015 - 09/17/2015

Previous experiments with Jews for Jesus had proven that the audience making it to their primary donation form were highly motivated. They were achieving a ~70% conversion rate for those making it to the page which led us to the conclusion that we were losing some low hanging fruit along the path to a donation.

Based upon these conclusions, we decided to run some experiments to find ways to increase the amount of traffic making it to the donation page. Our first attempt was focused solely on the home page where we added some additional donation buttons but did not see any statistical change to the amount of traffic being directed.  From this, we learned we needed to broaden our focus beyond just the homepage.

For this experiment, we rearranged the header of the website to add an additional donate button with two different calls to action. We expanded the targeting of the test so that it included the newsletter and RealTime sections of their website which are frequented by the target audience for donor acquisition.

Research Question

Which website header design will lead to the most donations?

Design

C: Only link in navigation
T1: "Make my Gift" Button
T2: "Donate Today!" Button

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Only link in navigation 2.0%
T1: "Make my Gift" Button 3.5%74.0% 96.5%
T2: "Donate Today!" Button 1.9%-6.0% 15.6%

This experiment has a required sample size of 1,066 in order to be valid. Since the experiment had a total sample size of 3,092, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 74.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The “Make my Gift” button added to the header increased the traffic to the donation page by 39%. This increase in traffic resulted in a 74% increase in total donations since there was an increased number of visitors and they were more highly qualified based upon the button they had clicked.

What is interesting is that the “Donate Today!” button ended up decreasing the amount of traffic being driven to the donation page by 9.6%. We learned that by putting the call to action in the context of the donor “Make my Gift” instead of a command “Donate Today!,” the donors were able to align better action and were more likely to click.


Experiment Documented by NextAfter

Question about experiment #2092

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.