Harvest Ministries

How presenting a problem to the reader affects donor conversion.

Experiment ID: #4537

Harvest Ministries

Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.

Experiment Summary

Timeframe: 07/11/2016 - 07/21/2016

Harvest wanted to test the motivation of their donors. To do this, they ran a test on their primary donation page. The control focused on the personal impact a donor can have. The headline of the control said, “Your gift helps reach the lost with the gospel of Jesus Christ.” The treatment didn’t initially focus on the impact a person could have, but rather presented a problem in the headline and then mentally walked a person through how Harvest Ministries could solve that problem with the donors help. The headline of the treatment said, “The World Needs the Gospel of Jesus Christ.”
Traffic going to the primary donation page was then split between the two variants for the test.

Research Question

Which copy approach would generate more donations?

Design

C: Your impact
T1: Presenting a Problem First

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Your impact 14.9%
T1: Presenting a Problem First 12.1%-18.8% 98.0%

This experiment has a required sample size of 1,097 in order to be valid. Since the experiment had a total sample size of 3,959, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 18.8% decrease in conversion rate
× 0% increase in average gift

Key Learnings

Our hypothesis was that by mentally walking a person through how giving a gift to Harvest Ministries would solve the problem we presented them with, we would increase donations. Our hypothesis was wrong. We saw a 18.8% decrease in donations with the treatment. What this tells us is that donors are more motivated by initially knowing the personal impact they can have than by presenting them with a problem and showing them later in the mental process of giving a gift that they can have an impact.


Experiment Documented by Courtney Gaines
Courtney Gaines is Vice President at NextAfter.

Question about experiment #4537

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.