How a modified value proposition affected campaign performance
Americans for Prosperity
Experiment Summary
Timeframe: 09/28/2015 - 09/30/2015
In the final days of the September money bomb for Hillsdale College, we were looking for ways to improve results. After reading the original copy on the page, we decided there were areas that we could add additional clarity to. Our hypothesis was that by adding the additional clarity, we could improve the understanding of the value proposition which would lead to increased conversion rate on the page.
Research Question
Which value proposition will have the biggest impact on revenue and conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 14.7% | ||
| T1: | Clarified Value Proposition | 14.6% | -0.5% | 4.6% |
This experiment has a required sample size of 1,997,521 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The modifications made to the copy on the page had no statistically significant impact on either revenue or conversion rate. In this situation, it would appear that the changes were not radical enough to make a major impact. It is also worth noting that at the end of a money bomb campaign such as this, the motivation of the visitors is often high enough that subtle changes will not have a major impact.