Adding personalization (but not first name) to subject line
Care Net
Experiment Summary
Ended On: 05/13/2019
This subject line test was performed in our weekly “weekend update” email.
Research Question
Does adding a personalization (“your community”) in a subject line increase open rates?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Pregnancy centers served 2 million people in 2017 | 11.3% | ||
| T1: | Your community pregnancy center helped serve 2 million people in 2017 | 11.9% | 5.7% | 99.5% |
This experiment has a required sample size of 19,030 in order to be valid. Since the experiment had a total sample size of 77,845, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
5.7% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
It DOES improve open rates by a small margin. We will be testing this again as well as adding it to our subject line formula.