Focus on the Family

How friction and anxiety influence donor conversion

Experiment ID: #2449

Focus on the Family

Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.

Experiment Summary

Timeframe: 10/05/2017 - 11/08/2017

Hillsdale College found that this treatment version did not significantly increase or decrease donor conversion on the ‘Constitution 101’ donation page. It’s apparent that this change in language was not dramatic enough to change the actions of the page visitor. Even though this test wasn’t statistically valid, we can still look at the trend of the test results for clues. We see that the donation page copy that focused more heavily on the premium offer, rather than the value proposition of Hillsdale College, was actually experiencing a lower conversion rate. This is interesting to remember as we develop more tests with donation pages that contain a premium offer.

Research Question

Will reducing the impact of the elements associated with friction and anxiety in the giving process improve conversion rate?

Design

C: Control
T1: Decreased Friction / Anxiety

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 3.9%
T1: Decreased Friction / Anxiety 4.1%5.5% 95.7%

This experiment has a required sample size of 64,821 in order to be valid. Since the experiment had a total sample size of 138,246, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 5.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment resulted in a 5.5% increase to donor conversion. While it is a relatively small percentage lift, since this test impacts all donation forms, it will result in a six-figure increase in revenue in the coming 12 months.

When doing a deeper analysis of the experiment, we found that the new treatment did not have as significant of an impact on traffic driven by email but was significantly more impactful for mobile traffic. For the mobile segment, it led to a 9.8% increase to overall conversion.

What this suggests is that the reduction of friction is not as important to a highly motivated audience (like email.) But for organic site visitors or those browsing on their phone, making the gift easier, was able to positively influence this lower motivated traffic.


Experiment Documented by NextAfter

Question about experiment #2449

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.