How a plain text email performed against an email with a logo banner Experiment ID: #12300

Focus on the Family

Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.

Experiment Summary

Ended On: 2/28/2019

Focus on the Family ran a year end campaign in 2018. They hypothesized that stripping down the “Last chance” emails to look more like Outlook or plain text emails would perform better than the control email which used large banner logos and video screenshots. They tested the outlook styles between two different signers.

Research Question

Will a plain text email receive more conversions than an email with all the bells and whistles?

Design

C: Branded
T1: Plain text

Results

Treatment Name Click Rate Relative Difference Confidence
C: Branded 0.04%
T1: Plain text 0.05% 21.4% 100.0%

This experiment has a required sample size of 425,658 in order to be valid. Since the experiment had a total sample size of 2,783,831, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    21.4% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The combined total of the plain text emails resulted in a 21.4% increase in engagement and donor conversion. This tells us that people give to people, not email marketing machines. Focus on the Family should continue to use plain text emails in future fundraising emails. 


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Courtney Brown

Courtney is an Optimization Associate at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.