How a more personal subject line affects open rate
CRISTA Ministries
Experiment Summary
Timeframe: 12/10/2014 - 12/18/2021
During the month of December, World Concern sent a series of emails for their calendar year-end fundraising campaign. In an effort to better understand what motivates their audience to open their emails and get more people to open them, they ran a subject line test. The control subject line was “Happy New Year!”. The treatment subject line was “I want to thank YOU”.
Research Question
Which subject line would motivate more people to open the email?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Happy New Year! | 10.9% | ||
| T1: | I want to thank YOU! | 12.9% | 19.0% | 99.9% |
This experiment has a required sample size of 1,901 in order to be valid. Since the experiment had a total sample size of 10,319, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
19.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment subject line, “I want to thank YOU” had an increase in opens by 19%. People are more motivated to open an email that sounds more personal and authentic than something that is more generic. The treatment caused intrigue and made a person wonder and want to know what the sender is thanking them for – resulting in more people opening the email.