How changing the sender name in a membership email affects the open rate
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 04/05/2021
In an effort to increase overall email engagement during a two-week high urgency annual campaign, we wanted to test different sender names to measure open and click response. We wanted to test the effect a different sender name would have on the open rate of an email. The control email was from “Maggy” while the treatment was from “Andrew.” The audience is accustomed to receiving membership emails from “Maggy” and weekly cultivation emails from “Andrew.”
Research Question
We believe that using a different sender name for the housefile audience will achieve a higher open rate .
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 21.4% | ||
| T1: | Treatment #1 | 21.2% | -0.7% | 47.3% |
This experiment has a required sample size of 515,839 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Though there was a .74% decrease in traffic this experiment does not validate because only a 47.3% confidence level was achieved. Further testing and a larger sample size would be needed in order to validate these results. We recommend sending from another person in the organization to see if that affects the open rate since the housefile is accustomed to seeing emails from both “Maggy” and “Andrew.”