How a radical redesign of an in-site ask affected clickthrough rate Experiment ID: #8993

Leadership Institute

Experiment Summary

Timeframe: 5/1/2018 - 5/15/2018

Leadership Institute had seen a large jump in donation traffic through the implementation of an end-of-article call-to-action on their CampusReform website. They wanted to test to see if improved copy and layout could increase clickthrough rate from the site and hopefully increase overall donation revenue. They created a treatment that added increased persuasive copy, a third-party quote, and statistics to bolster the value proposition. They also made a softer ask in the button, asking the reader to “learn more” rather than “support Campus Reform now”

Research Question

How will a radical redesign of an end-of-article call-to-action affect clickthrough rate?

Design

C: Control
T1: Treatment 1

Results

Treatment Name Click Rate Relative Difference Confidence
C: Control 0.11%
T1: Treatment 1 0.10% -8.1% 51.3%

This experiment has a required sample size of 1,021,322 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

Ultimately, there was no valid difference in clickthrough rate. After receiving several hundred impressions per CTA, the treatment showed a slight decrease in clickthroughs, but nowhere near validity. The team went back to the drawing board to try to find something more radical that would positively increase clicks.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.