How a plain text email performed against an email with a logo banner Experiment ID: #11055

Leadership Institute

Experiment Summary

Timeframe: 1/7/2016 - 1/21/2016

The Leadership Institute wanted to know if including an LI logo banner in an email would make a better impact than a plain text email. They decided to test it.

Research Question

Does an LI banner generate more clicks than an email without an LI banner in it?

Design

C: with LI banner
T1: no LI banner

Results

Treatment Name Click Rate Relative Difference Confidence
C: with LI banner 14.4%
T1: no LI banner 16.3% 12.6% 97.3%

This experiment has a required sample size of 3,002 in order to be valid. Since the experiment had a total sample size of 7,641, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    12.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The plain text email saw a 12.6% lift in traffic when tested against the banner email. Moving forward, the Leadership Institute should use plain text emails in all of their sends.


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.