How the sender’s title impacted open and clickthrough rates on a weekly newsletter
The Fund for American Studies
Experiment Summary
Ended On:
As a part of continuous experimentation with increasing engagement from their weekly newsletter, The Fund for American Studies had recently tested into switching senders, but wanted to experiment with whether or not it was important to keep the sender’s title/department as a part of their name in following emails.
Research Question
How will including the sender’s org/title in the sender name field impact email open, click and unsubscribe rates?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | From: Kristin Underwood | 15.7% | ||
| T1: | From: Kristin, TFAS Communications | 15.4% | -2.1% | 83.5% |
This experiment has a required sample size of 92,251 in order to be valid. Since the experiment had a total sample size of 92,748, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
After 4 weekly sends were combined together, we could not validate a lift (or decrease) in any of the metrics we were tracking (open rate, clickthrough rate, or unsubscribe rate).
We did see a slight (albeit not valid) lift in the open rate (and each subsequent metric) when we did not use the sender’s org/title within the “sender name” field, but even so, the difference between the control/treatment was so negligible (at/below 3% across all three metrics), it’s safe to assume that it’s of no great consequence (or threat) to use either variation.
Our recommendation is to always err on the side of being a real person speaking to one individual recipient, and therefore would default to not including the org name/title as a part of the sender name (because most people don’t send emails with that included), however, there is no data in this particular study that would substantiate a significant gain in going that route.