Will an emoji in the subject line help increase open rate?
NextAfter
Experiment Summary
Timeframe: 02/14/2020 - 02/28/2020
People use emoji’s all the time and we get asked how they can or should be used in emails and subject lines. We didn’t have any good experiments available to justify our thinking that it isn’t a worthwhile test so decided to test it for ourselves. We were sending a digest email on Valentines day so tied in the subject line to ‘love’ for the control and for the treatment we used a heart emoji in its place.
Research Question
Will an emoji in the subject line help increase open rate?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Fundraising Content You’ll Love… see what I did there… | 42.4% | ||
| T1: | Fundraising Content You’ll ❤️… see what I did there… | 42.1% | -0.9% | 25.0% |
This experiment has a required sample size of 137,194 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Short answer: nope, didn’t do anything. With a 25% level of confidence, the emoji didn’t seem to have any impact on the likelihood of a subscriber to open.