Formatted vs. Unformatted Experiment ID: #6930

Jewish Voice Ministries

JVM’s mission is two-fold: Proclaiming the Gospel to the Jew first, and also to the Nations (Romans 1:16), and equipping the Church—providing education about the Hebraic roots of Christianity, the Church’s responsibility to Israel and the Jewish People, and how to share Messiah with the Jewish People.

Experiment Summary

Timeframe: 6/29/2017 - 7/3/2017

We send a weekly email out to roughly 200,000. It comes from our CEO and is new oriented. His full name is in the FROM and the subject lines are generally like a news headline. The emails are formatted and branded.

Research Question

Will stripping the formatting from an email increase the Opens, Clicks, and conversions (donations) when tested against our standard branded email?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS


Treatment Name Open Rate Relative Difference Confidence Average Gift
C: Standard Branded Email 14.6% $45.00
T1: No Formatting 25.7% 75.3% 100.0% $65.00

This experiment has a required sample size of 102 in order to be valid. Since the experiment had a total sample size of 200,000, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    75.3% increase in traffic
× 0% increase in conversion rate
× 44.4% increase in average gift
153.2% increase in revenue

Key Learnings

This was by far our most dramatic experiment of late, in terms of the results. Drastically more opens and clicks and decent increase in donations, both in number and value, when measured against the control. The less formal approach seemed to appeal to our readers. I believe they felt it was more personal and therefore more valuable and trustworthy, even though it lacked the normal branding they are accustomed to.

Our opens for the unformatted one were nearly twice as many, as were click-throughs and donations and transactions were 50% higher. It was a dramatic result and we want to test out some of the micro details to see if it was a combination of things or one specific thing.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Experiment Documented by...

Mark Santiago

Mark is . If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.