How email design impacted opens with a prospecting email list
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 05/23/2018 - 05/25/2018
ADF was using their existing email welcome series to “warm up” the purchased prospecting list from Axciom. The open rate of this list was significantly low compared to the standard house file emails so we decided to test some deliverability tactics to improve this metric. We had found the sender did not play a significant roll in increasing the opens so we decided to test the style of the email. The hypothesis was that a simpler email would not be placed in the “Promotions” tab of systems like Gmail and Yahoo which would increase the likelihood of being opened.
Research Question
Will a simpler email increase the likelihood of the email being opened?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Designed Email | 2.4% | ||
| T1: | Plain Text Email | 2.5% | 3.1% | 48.9% |
This experiment has a required sample size of 328,891 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
We did not see a statistically significant different in open rate between both the designed and plain text emails. This is actually beneficial for ADF as their standard emails to their subscriber base have the more “designed” style which means this email file will fit well within their audience.