How the subject line capitalization impacts open rate
Illinois Policy Institute
Experiment Summary
Timeframe: 04/19/2017 - 04/28/2017
The Illinois Policy Institute has historically used all lower case subject lines when sending emails. This is the result of past experiments where this tactic proved to increase open rates. However, over time, an audience’s behaviors can shift as they grow accustomed to a certain tactic or the audience changes. To ensure that this is still the best tactic, it was decided to test it again to validate its continued effectiveness.
Research Question
Does using proper case increase the open rate of an email?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | happy birthday | 23.2% | ||
| T1: | Happy Birthday! | 19.7% | -15.1% | 100.0% |
This experiment has a required sample size of 1,053 in order to be valid. Since the experiment had a total sample size of 18,988, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
15.1% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
It turns out that the lowercase subject line continue to provide the greatest open rate for the file. In this situation, the proper case subject line lowered open rate by 15%. It is recommended to continue to utilize the lowercase subject lines moving forward.