How removing the organization from a sender name affected click rates
Texas Public Policy Foundation
Experiment Summary
Ended On: 04/05/2022
A new president was announced within the organization, meaning their email list would be receiving emails from a sender name they may not recognize. To mitigate confusion and prevent disengagement, unsubscribes, and SPAM complaints, the organization name was appended to the sender name on emails sent to their list. We wanted to know how this was affecting click rates and decided to test if we would see a lift in click rates by once again removing the organization from the sender name. For this experiment, we ran an A/B test with traffic split 50/50.
Research Question
We believe that removing the organization from the sender name for email recipients will achieve a higher click rate.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.09% | ||
| T1: | Treatment #1 | 0.18% | 101.5% | 84.7% |
This experiment has a required sample size of 12,683 in order to be valid. Since the experiment had a total sample size of 13,525, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
This experiment produced a 101.5% lift in click rates, with an 84.7% level of confidence. Despite being unable to validate this lift, the results provide a strong indication that there may be a link between sender name and click rates. This is especially interesting because the sender name is most often thought to affect open rates, but its impact on click rates is less commonly documented.
These results reinforce the knowledge that the more personal and human your outreach (especially via email), the better chance you have of connecting with donors and inspiring them to act. Whenever possible, organizations should seek to humanize their outreach and appeals—this includes ensuring that emails are sent from real people within an organization and should appear in an inbox as much like a personal email as possible. The results also show how even a small change made to one part of a campaign can have a significant affect on downstream performance.