How changing the sending domain impacted email engagement rates
Americans for Prosperity
Experiment Summary
Timeframe: 07/14/2020 - 07/19/2020
Americans for Prosperity had been running a number of petition and survey acquisition offers and produced hundreds of thousands of new email addresses, donors, and revenue through this acquisition process. However, with such a large volume of emails coming through these types of offers, over time they noticed their email engagement rates (deliveries, opens, and clicks) going down.
They had two email sender domains available to them, so instead of sending from the one they’ve been primarily using, they decided to test sending an email from the secondary email sender domain to monitor email engagement rates against the primary domain.
Research Question
Will switching email sender domains increase email engagement rates?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | "Tim Phillips" | 7.4% | ||
| T1: | "Tim Phillips" | 10.1% | 37.2% | 100.0% |
This experiment has a required sample size of 817 in order to be valid. Since the experiment had a total sample size of 70,086, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
37.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a level of confidence of 100% — we observed a 37.2% increase in email open rates when using the secondary sender domain.
Subsequently, we observed a +49.8% increase in email clickthrough rate for the email used within the test, as well.
If email deliverability is an issue for you, perhaps creating a new sender domain may be a technique you can use to increase email deliverability and engagement over time.
When creating a new email sender domain, it’s “reputation” must be built up over time. To do this, it requires that the list be segmented into two types: (1) engaged emails, and (2) disengaged emails. For at least 3-months, this new sender domain should only send emails to the “engaged” list. This will help it establish itself as being of “good quality” when evaluated by mail servers.
This will lead to good inbox rates for all emails sent over time.
While this “warm-up” period of time is underway, continue to send your emails from the original sender domain, but only to the “disengaged” portion of the list.
You can send the same email creative, but it’s advised that you lead with emailing the “engaged” file first, wait at least an hour, then send the same email to the “disengaged” list from the original email sender domain.
Ideally, recent opens/clicks from the “disengaged” list should move those contacts to the “engaged” segment (and vise versa for people who continue not to open/click if they were originally placed into the “engaged” list).
All of this will allow you to warm up a different email sender domain and could yield increased results in email delivery, email inboxing rates, which will subsequently lead to increases in open and click rates, as well.