How specificity in a subject line impacts open rate
Care Net
Experiment Summary
Timeframe: 11/11/2020 - 11/24/2020
Care Net was sending a cultivation email to their file and wanted to test the element of specificity in their subject line. The control was “You should read this with your family” and the treatment was, “You should read this with your kids”.
Research Question
Will specificity in the subject line increase conversion?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | You should read this with your family | 32.7% | ||
| T1: | You should read this to your kids | 33.2% | 1.5% | 79.3% |
This experiment has a required sample size of 67,791 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
We weren’t able to validate the results, but we did see a directional lift from the treatment. This tells us that the specificity from the treatment minimally helped increase engagement, but wasn’t a significant enough difference to have a greater impact.