How oversimplifying an open letter’s readability impacted revenue | NextAfter

How oversimplifying an open letter’s readability impacted revenue

Results

This experiment has a required sample size of in order to be valid. Since the experiment had a total sample size of , and the level of confidence is not above 95% the experiment results are not valid.


The Missionary Oblates of Mary Immaculate

The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.

Click on each one to dive into the details and why it’s important for your online fundraising.

Experiment Summary

Ended On: 6/9/2020

Missionary Oblates send out a monthly evangelize newsletter. They use the simpler format on all other emails that come from their organization as a way of creating a one-to-one relationship with the subscribers. They want to see if this same concept would apply to the stylized newsletter sent out each month. To do this, instead of breaking up all the stories, they put the entire first story into the email followed by the other links. The goal was the measure the impact on engagement (including giving).

Research Question

Would creating a simpler format for our Evangelize emails increase donations?

Design

C: Control
T1: Treatment #1

Result

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.02%
T1: Treatment #1 0.05% 171.4% 98.1%

This experiment has a required sample size of 26,789 in order to be valid. Since the experiment had a total sample size of 77,274, and the level of confidence is above 95% the experiment results are valid.

Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #17719

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.