IAS - General Email - February 2021 | NextAfter
e3 Partners

IAS – General Email – February 2021

Experiment ID: #63519

e3 Partners

Experiment Summary

Timeframe: 02/23/2021 - 03/09/2021

Control was our normal, fully designed newsletter format from I Am Second.

Treatment #1 was a fully plain-text email coming from Doug.

Research Question

We believe that changing sender and using plain-text for our general list will achieve a higher open rate.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Open Rate Relative Difference Confidence
C: Control 25.7%
T1: Treatment #1 42.8% 66.6% 100.0%

This experiment has a required sample size of 59 in order to be valid. Since the experiment had a total sample size of 22,590, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    66.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

Open rate was higher on Treatment #1, but Click-through late was much higher on the control. We will continue testing this to see if it changes.


Experiment Documented by NextAfter

Question about experiment #63519

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.