How reducing specificity affects email acquisition
Hoover Institution
Experiment Summary
Timeframe: 10/01/2015 - 11/24/2015
The Hoover Institution launched a campaign to grow the subscriber base of the Hoover Daily Report, their flagship policy newsletter. They drafted landing page copy that highlighted the authority of their research fellows (who provide content for the Daily Report), but wanted to test a much simpler value proposition inspired by another email newsletter. They hypothesized that this new simple headline would increase email acquisition rate.
Research Question
Will a simpler headline clarify the value proposition and increase email acquisition rate?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Authority | 60.5% | ||
T1: | Simplified Headline | 64.3% | 6.2% | 92.7% |
This experiment has a required sample size of 1,290 in order to be valid. Since the experiment had a total sample size of 2,161, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
Though it was only a slight lift, there was enough difference to achieve statistical validity. The simpler headline produced a 6% lift in email conversion — which tells us that we don’t always have to be selling. Many times, our prospects can be attracted to subscribe by clarity rather than perceived authority.
Question about experiment #2697
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.