How a radical redesign of an in-site ask affected clickthrough rate
Leadership Institute
Experiment Summary
Timeframe: 05/01/2018 - 05/15/2018
Leadership Institute had seen a large jump in donation traffic through the implementation of an end-of-article call-to-action on their CampusReform website. They wanted to test to see if improved copy and layout could increase clickthrough rate from the site and hopefully increase overall donation revenue. They created a treatment that added increased persuasive copy, a third-party quote, and statistics to bolster the value proposition. They also made a softer ask in the button, asking the reader to “learn more” rather than “support Campus Reform now”
Research Question
How will a radical redesign of an end-of-article call-to-action affect clickthrough rate?
Design
Results
Treatment Name | Click Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Control | 0.11% | ||
T1: | Treatment 1 | 0.10% | -8.1% | 51.3% |
This experiment has a required sample size of 1,021,322 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Ultimately, there was no valid difference in clickthrough rate. After receiving several hundred impressions per CTA, the treatment showed a slight decrease in clickthroughs, but nowhere near validity. The team went back to the drawing board to try to find something more radical that would positively increase clicks.
Question about experiment #8993
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.