How increasing the size of the text on an offer landing page did not impact the email registration rate
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 04/12/2019 - 04/15/2019
When The Heritage Foundation launched a new offer, we were not achieving the email registration rate that we would normally see. So, in an attempt to boost the email registration rate, we launched a simple experiment to boost the size of the text across both desktop and mobile versions to see if readability was prohibiting us from achieving the results we wanted.
Research Question
Will improved readability increase the email registration rate for the Religious Freedom offer?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | control - smaller text | 28.6% | ||
T1: | treatment - larger text | 26.2% | -8.3% | 80.9% |
This experiment has a required sample size of 2,691 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
With an 80.9% level of confidence, we found that the large text actually decreased the email registration rate by -8.3%.
We will now turn our testing towards the message itself, having eliminated something structural with the page as a variable that is limiting our effectiveness.
Question about experiment #14307
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.