How shortening the length of content impacted email acquisition Experiment ID: #9320
Americans for Prosperity
Timeframe: 7/18/2018 - 7/25/2018
The Open Letter to the Senate had been a high performing email acquisition campaign for Americans for Prosperity for several weeks now. With an email acquisition rate higher than 60%, we had found it difficult to improve results through optimization.
Based upon the high motivation of the visitors coming to the page, we decided to see if simplifying the page copy could improve the results. The hypothesis was that, given the high motivation of visitors, they did not need as much value proposition to make the conversion. Our goal was to maintain the value expressed through the copy but shorten it up so that new visitors would find it easier to convert.
Will shortening the value proposition on the page increase email acquisition rates?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 12,285 in order to be valid. Since the experiment had a total sample size of 25,474, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 1.9% increase in conversion rate
× 0% increase in average gift
It would appear that the length of the page did impact the likelihood of a highly motivated audience to convert, albeit only a minimal amount. We were able to statistically prove that the reduced length of copy increased conversion rate by 1.9%.
This is a unique situation where the motivation of the visitors was so high, they did not need much convincing. When audiences are less motivated, when the conversion rate is not as high, increasing the perceived value of the offer (even if it makes the page longer) has proven to be a more effective method for boosting conversion than getting the form “above the fold.” The lift of only 1.9% is evidence of just how minimally impactful the friction associated with copy length is to the conversion process.