How changing the sender name affects the open rate of an email
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 04/05/2021
In an effort to increase overall email engagement during a two-week high urgency annual campaign, we wanted to test sender names to measure open and click response. We tested two different names from the organization in order to see how each name would affect the open rate of an email. We used “Maggy Smith” for the control email and “Andrew McIndoe” for the treatment. “Andrew” sends a weekly cultivation email while “Maggy” is the usual sender for membership card appeals.
Research Question
We believe that changing the sender name for the housefile audience will achieve a greater open rate .
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 24.1% | ||
| T1: | Treatment #1 | 24.4% | 1.0% | 66.9% |
This experiment has a required sample size of 221,062 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Though we saw a 1% increase in traffic, this test did not validate because it resulted in a confidence level of 66.8%. However, a retest with a larger sample size is recommended in order to validate results. There was also no statistical difference in the click thru rates for each email. We recommend potentially using another person’s name to see if it would significantly increase open rates since the housefile is used to seeing both “Andrew” and “Maggy” in their inbox.