How a celebrity signer affects open rates
Leadership Institute
Experiment Summary
Timeframe: 05/10/2018 - 05/10/2018
The Leadership Institute has access to many high profile people in the conservative movement. They wanted to see if mentioning these people would impact their online presence. They decided to test a subject line that states Mike Huckabee has involvement with LI vs a vague subject line about something that makes students freeze when they hear it.
Research Question
Will a subject line that mentions a conservative celebrity increase open rates?
Design
Results
Treatment Name | Open Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Mike Huckabee Hosts Ll's own | 18.3% | ||
T1: | Students freeze up when they hear this | 20.0% | 9.5% | 99.9% |
This experiment has a required sample size of 3,907 in order to be valid. Since the experiment had a total sample size of 23,284, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
9.5% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The subject line that included Mike Huckabee garnered an 18.3% open rate, while the vague subject line about students earned a 20% open rate. Though only a small lift of 9.5%, this experiment tells us that LI email recipients care more about what is being discussed on campus than about a conservative celebrity hosting an LI employee on their show. However, these results could change depending on the celebrity signer.
Question about experiment #10307
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.