How a “forwarded” email impacts conversion
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Ended On: 02/28/2019
Focus on the Family ran a year end campaign in 2018. They hypothesized that sending an email that looked like a “FWD” would increase engagement and conversion. They decided to test this against an email with a subject line that didn’t have “FWD” in it.
Research Question
How will a forward email impact conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.01% | $279.82 | ||
| T1: | Forward email | 0.01% | -4.5% | 26.6% | $327.39 |
This experiment has a required sample size of 30,845,701 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
This experiment is not valid because the level of confidence is only 51.2%. Having a forwarded email has no impact on conversion. The reason for this is because motivation is so high of the year. In the context of a year end campaign, a forwarded email has no impact on donor conversion.