How personal subject line tone affects fundraising email open rate
Harvest Ministries
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Experiment Summary
Timeframe: 04/01/2016 - 05/02/2016
Harvest Ministries was sending a mid-month update on their fundraising appeal for the month. The offer was a book by Joni Earackson Tada, and the control subject line was all about the offer. We’ve seen that when an offer is obviously marketing, our prospects become resistant to it and don’t open the email. So we tested an alternate subject line that sounded like one a friend might actually write: “An idea for you”.
Research Question
Will a more personal subject line increase open rate?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Sales Tone | 22.2% | ||
| T1: | Personal Tone | 24.0% | 7.9% | 100.0% |
This experiment has a required sample size of 4,439 in order to be valid. Since the experiment had a total sample size of 81,181, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
7.9% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment lifted open rate by 8%. There is a possibility that these openers might be less motivated, since the control subject line was clearly an offer, so we must make sure to measure all the follow-on metrics to ensure that we aren’t leaving conversions on the table.