Harvest Ministries

How adding a testimony to an email appeal can impact click-through.

Experiment ID: #2979

Harvest Ministries

Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.

Experiment Summary

Timeframe: 06/01/2017 - 06/05/2017

The treatment with the testimony increased clicks by 27.6%. While these are great results and want to get more people to the donation pages, the increased clicks also resulted in an increase in overall revenue. What we can take away from this test is that by adding the testimonial to the email copy, we were able to strengthen the value proposition motivating more people to click through to the donation page and give more generously.

Research Question

Would adding a testimony in the email copy increase click-through?

Design

C: CTRL: No Testimony
T1: TRT - With Testimony

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: CTRL: No Testimony 1.7%
T1: TRT - With Testimony 2.2%27.6% 100.0%

This experiment has a required sample size of 6,674 in order to be valid. Since the experiment had a total sample size of 61,041, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    27.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

Leading up to Harvest Ministries large-scale evangelism outreach, they ran a high-urgency fundraising campaign. This fundraising campaign consists of multiple emails being sent over a short period of time to reach a specific financial goal. In an effort to optimize the emails as the campaign goes on, they ran a series of experiments within the emails and landing page. One of these tests was the use of a personal testimony from someone who had been to one of Harvest’s large-scale evangelism events. This personal testimony was added to the email copy for the treatment. They A/B split their file.


Experiment Documented by Courtney Gaines
Courtney Gaines is Vice President at NextAfter.

Question about experiment #2979

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.