How reducing specificity affects email acquisition Experiment ID: #2697
Timeframe: 10/1/2015 - 11/24/2015
The Hoover Institution launched a campaign to grow the subscriber base of the Hoover Daily Report, their flagship policy newsletter. They drafted landing page copy that highlighted the authority of their research fellows (who provide content for the Daily Report), but wanted to test a much simpler value proposition inspired by another email newsletter. They hypothesized that this new simple headline would increase email acquisition rate.
Will a simpler headline clarify the value proposition and increase email acquisition rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 1,290 in order to be valid. Since the experiment had a total sample size of 2,161, and the level of confidence is not above 95% the experiment results are not valid.
Though it was only a slight lift, there was enough difference to achieve statistical validity. The simpler headline produced a 6% lift in email conversion — which tells us that we don’t always have to be selling. Many times, our prospects can be attracted to subscribe by clarity rather than perceived authority.