Leadership Institute

How a radical redesign of an in-site ask affected clickthrough rate

Experiment ID: #8993

Leadership Institute

Experiment Summary

Timeframe: 05/01/2018 - 05/15/2018

Leadership Institute had seen a large jump in donation traffic through the implementation of an end-of-article call-to-action on their CampusReform website. They wanted to test to see if improved copy and layout could increase clickthrough rate from the site and hopefully increase overall donation revenue. They created a treatment that added increased persuasive copy, a third-party quote, and statistics to bolster the value proposition. They also made a softer ask in the button, asking the reader to “learn more” rather than “support Campus Reform now”

Research Question

How will a radical redesign of an end-of-article call-to-action affect clickthrough rate?

Design

C: Control
T1: Treatment 1

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Control 0.11%
T1: Treatment 1 0.10%-8.1% 51.3%

This experiment has a required sample size of 1,021,322 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

Ultimately, there was no valid difference in clickthrough rate. After receiving several hundred impressions per CTA, the treatment showed a slight decrease in clickthroughs, but nowhere near validity. The team went back to the drawing board to try to find something more radical that would positively increase clicks.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #8993

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.