Harvest Ministries

How a personalized email affects donor conversion.

Experiment ID: #4647

Harvest Ministries

Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.

Experiment Summary

Timeframe: 08/03/2016 - 08/05/2016

Harvest Ministries was sending their monthly fundraising email for the month of August and wanted to test the motivation behind people’s donations. All three email variations that were sent and tested offered a monthly premium, a book by Greg Laurie call, Tell Someone. Greg Laurie was the sender of all variations.

The control email was written and laid out how Harvest traditionally would send their monthly email. They developed two other treatments to test against the control. For the first treatment, they tested using urgency as a motivating factor to give a donation. The layout and design of the email was similar to the control, but the call-to-action had copy that created a sense of urgency and immediacy (“Time is running out, get this resource now before it’s too late.”).  For the second treatment, they tested a more personal approach to the email. They stripped down the traditional marketing email template to make it look like it was an Outlook email personally from Greg and changed the tone of the copy and email to be more casual and conversational as if Greg Laurie had sat down to personally write the donor this appeal.

Research Question

What motivates donors to give a donation?

Design

C: Control
T1: Urgency
T2: Personal

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.07%
T1: Urgency 0.10%36.3% 80.5%
T2: Personal 0.16%116.3% 100.0%

This experiment has a required sample size of 17,568 in order to be valid. Since the experiment had a total sample size of 123,249, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 116.3% increase in conversion rate
× 18.9% decrease in average gift
75.4% increase in revenue

Key Learnings

While the urgency focused email didn’t quite validate, both treatments increased donations from the control. The urgency email had an increase in donations by 36.3% and the personalized email treatment had an increase in donations by 116%. The personalized email results are 100% valid. What this tells us is that people are more motivated to give if they know and feel that Greg is personally asking them. It wasn’t just the copy that contributed to this lift. Having the email not look like a traditional email, people didn’t feel like Greg was marketing to them. People want to feel like they are being personally engaged and that the appeal is only written to them from Greg. Moving forward it will be important to send emails that are more high-touch like this winning treatment and look and feel like a “normal” email a person would send.

 


Experiment Documented by Courtney Gaines
Courtney Gaines is Vice President at NextAfter.

Question about experiment #4647

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.