Leadership Institute

How a celebrity signer affects open rates

Experiment ID: #10307

Leadership Institute

Experiment Summary

Timeframe: 05/10/2018 - 05/10/2018

The Leadership Institute has access to many high profile people in the conservative movement. They wanted to see if mentioning these people would impact their online presence. They decided to test a subject line that states Mike Huckabee has involvement with LI vs a vague subject line about something that makes students freeze when they hear it. 

Research Question

Will a subject line that mentions a conservative celebrity increase open rates?

Design

C: Mike Huckabee Hosts Ll's own
T1: Students freeze up when they hear this

Results

 Treatment NameOpen RateRelative DifferenceConfidence
C: Mike Huckabee Hosts Ll's own 18.3%
T1: Students freeze up when they hear this 20.0%9.5% 99.9%

This experiment has a required sample size of 3,907 in order to be valid. Since the experiment had a total sample size of 23,284, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    9.5% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The subject line that included Mike Huckabee garnered an 18.3% open rate, while the vague subject line about students earned a 20% open rate. Though only a small lift of 9.5%, this experiment tells us that LI email recipients care more about what is being discussed on campus than about a conservative celebrity hosting an LI employee on their show. However, these results could change depending on the celebrity signer. 


Experiment Documented by NextAfter

Question about experiment #10307

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.