How creating a simpler email format increases donations and revenue Experiment ID: #21354
The Missionary Oblates of Mary Immaculate
The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.
Ended On: 6/9/2020
Missionary Oblates send out a monthly evangelize newsletter. They use the simpler format on all other emails that come from their organization as a way of creating a one-to-one relationship with the subscribers. They want to see if this same concept would apply to the stylized newsletter sent out each month. To do this, instead of breaking up all the stories, they put the entire first story into the email followed by the other links. The goal was the measure the impact on engagement (including giving).
Would creating a simpler format for our Evangelize emails increase donations?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 26,789 in order to be valid. Since the experiment had a total sample size of 77,274, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 171.4% increase in conversion rate
× 0% increase in average gift
We ended up with a 171% increase in conversion rate and more than tripled our revenue! We learned that adding the story with a straight donation link increased donations significantly. The email was also more personal. Based off of these results we will continue using the simpler format with less links.