How the sender influences email open rates
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Ended On: 03/26/2018
Alliance Defending Freedom was sending a fundraising email as part of their March campaign. We had created a series of optimizations to test on the email, the first of which was who the email came from. Our goal for the whole email was to make it feel more personalized; like it was coming from an individual and not a faceless organization.
Research Question
Will sending the email from an individual (Kristen Waggoner) have a positive impact on open rate?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Alliance Defending Freedom | 17.8% | ||
| T1: | Kristen Waggoner | 22.4% | 25.6% | 100.0% |
This experiment has a required sample size of 594 in order to be valid. Since the experiment had a total sample size of 128,204, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
25.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
We saw a 26% increase to the open rate by sending from an individual. All other aspects of the email envelope were the same (subject line and preview text) which means the email sent from an individual was able to garner much better engagement than the one sent from the organization. It is worth pointing out that this may be the result of the sender being seen as a “new shiny object.” If this tactic is used as the new control moving forward, it may begin to lose its impact over time. This is something we will want to iteratively test over time.