How adding a premium offer to a daily email affects click rate Experiment ID: #9330

The Missionary Oblates of Mary Immaculate

The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.

Experiment Summary

Ended On: 7/25/2018

The Missionary Oblates send out a daily prayer email each day to a segment of their housefile. They wanted to test whether adding a complementary resource donation opportunity would increase click rate in the emails and encourage more people to give from the daily emails. The rest of the email layout remained the same, with a section of value prop about the premium and an image of the book in the middle of the email.

Research Question

Does adding a premium offer to a daily email affect click rate?

Design

C: No Hardcover Book
T1: With Hardcover Book

Results

Treatment Name Click Rate Relative Difference Confidence
C: No Hardcover Book 0.83%
T1: With Hardcover Book 2.2% 166.9% 100.0%

This experiment has a required sample size of 604 in order to be valid. Since the experiment had a total sample size of 37,292, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    166.9% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment email with the daily prayer hardcover book increased click rate by 166%, and also received 7 donations for $208. This test shows that this segment of the email file, that is highly cultivated each day, is open to a donation opportunity when it is tied to a premium that aligns with the content of the email they’re getting each day.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Allison Jones

Allison is an Optimization Associate at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.