CRISTA Ministries

How simplifying an email template improved open rates.

Experiment ID: #7586

CRISTA Ministries

Experiment Summary

Ended On: 11/09/2018

In reviewing historical promotional emails for the Global Gift Guide, it was clear that the organization has always leaned upon well designed and graphic-heavy images to promote the gift catalog. Our research on the subject has typically yielded improved results when stripping email templates down to “plain text” style emails (still with complete tracking and HTML-based) to appear as if they were written from an individual.

We wanted to experiment with emails in 2018 to verify if this approach would also work for World Concern.

Research Question

Will a simple plain text style email improve open and/or clickthrough rates?

Design

C: Control
T1: Treatment #1

Results

 Treatment NameOpen RateRelative DifferenceConfidence
C: Control 12.6%
T1: Treatment #1 17.0%35.0% 98.1%

This experiment has a required sample size of 498 in order to be valid. Since the experiment had a total sample size of 1,413, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    35.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 98% confidence score, this experiment validated that “plain text” style emails improved open rates by 35% for World Concern.

Technologically speaking, highly stylized and image-heavy emails are typically “caught” by the email server that receives them before sending them to the intended recipient, and have a higher chance of being codified as a spam message. This plain text style template not only has shown to improve engagement rates because people give to people, not organizations, it also typically yields much higher open and click volumes due to the higher inboxing rates.

When possible, always find a way to send message from a person at your organization to your donors.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #7586

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.