How more personal phrasing in a reply email impacts opens
NextAfter
Experiment Summary
Timeframe: 11/05/2019 - 11/06/2019
For an upcoming full-day certification workshop, we sent an email asking people to reply with a ‘yes’ or ‘no’ on whether or not they are interested in attending as a way to a) get commitment but also b) get a sense of numbers to plan the event properly. We wanted to see if being more direct in the subject line by telling them they will be asked to answer a question or being a bit more personal and casual subject line that isn’t as direct about a question would increase opens.
Research Question
Will a more direct subject line increase opens and responses?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Can You Please Answer One Question For Me? | 32.3% | ||
| T1: | Can You Please Let Me Know If You Want to Come? | 41.3% | 27.8% | 87.6% |
This experiment has a required sample size of 222 in order to be valid. Since the experiment had a total sample size of 271, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
While not statistically valid, the more personal, casual approach and not letting them know about the question seems to be a better approach but more testing is needed.