How adding a testimony to an email appeal can impact click-through.
Harvest Ministries
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Experiment Summary
Timeframe: 06/01/2017 - 06/05/2017
The treatment with the testimony increased clicks by 27.6%. While these are great results and want to get more people to the donation pages, the increased clicks also resulted in an increase in overall revenue. What we can take away from this test is that by adding the testimonial to the email copy, we were able to strengthen the value proposition motivating more people to click through to the donation page and give more generously.
Research Question
Would adding a testimony in the email copy increase click-through?
Design
Results
Treatment Name | Click Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | CTRL: No Testimony | 1.7% | ||
T1: | TRT - With Testimony | 2.2% | 27.6% | 100.0% |
This experiment has a required sample size of 6,674 in order to be valid. Since the experiment had a total sample size of 61,041, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
27.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
Leading up to Harvest Ministries large-scale evangelism outreach, they ran a high-urgency fundraising campaign. This fundraising campaign consists of multiple emails being sent over a short period of time to reach a specific financial goal. In an effort to optimize the emails as the campaign goes on, they ran a series of experiments within the emails and landing page. One of these tests was the use of a personal testimony from someone who had been to one of Harvest’s large-scale evangelism events. This personal testimony was added to the email copy for the treatment. They A/B split their file.
Question about experiment #2979
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.