How emotional language, clarity and authority affect ad clickthrough rate Experiment ID: #7843

Leadership Institute

Experiment Summary

Timeframe: 10/2/2017 - 11/9/2017

Leadership Institute recently re-launched a course about the history and principles of conservatism, called Conservatism 101. As they prepared to market this course through Facebook, they developed several different treatments for the advertisements. The first treatment posed a question: “What are conservatives actually trying to conserve?”, which implied that the course would answer that question. The second treatment introduced emotional language that outlined the threat to conservatism and made a call for conservatives to rediscover their principles. This treatment also included a call to action (CTA) in the top copy: “Start today by enrolling in Conservatism 101 now.” The third treatment took a similar tack with emotional language, but sought to align with the mindset of the reader by declaring the threat to be invalid. This treatment heightened the clarity and authority around the course offer, introducing a list of featured speakers, and introduced line breaks to better isolate the CTA: “Activate your free course today and discover—or rediscover—the principles and beauty of conservatism.” The final treatment used the same copy as the third treatment, but introduced a picture of Mark Levin with the CTA “Join Mark Levin in this new free course: Conservatism 101.”

Since Facebook is able to serve multiple ads simultaneously and gather data on each, they launched this A/B/C/D test to determine which ad produced the most clicks.

Research Question

How does emotional language, clarity and authority affect clickthrough rate?


C: Question-focused ad
T1: Challenger copy
T2: Separate interruptor line
T3: Levin ad


Treatment Name Click Rate Relative Difference Confidence
C: Question-focused ad 0.82%
T1: Challenger copy 0.89% 9.6% 61.3%
T2: Separate interruptor line 1.3% 56.7% 100.0%
T3: Levin ad 1.7% 108.7% 100.0%

This experiment has a required sample size of 2,591 in order to be valid. Since the experiment had a total sample size of 112,921, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    108.7% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

After a few days, LI had enough data to determine a winner. Both the third and fourth treatment produced a clear, statistically significant increase in clickthrough rate over the other treatments. Treatment 3 produced a 56.7% increase over the baseline due to a combination of increased clarity and authority as well as visual clarity around the call-to-action. Treatment 4 produced the largest increase of all—a 108.7% increase over the baseline due to the copy adjustments from treatment three and the new image and embedded CTA, which reinforced the authority of the top copy.

In their next experiment, Leadership Institute isolated the top two performing ads to determine which one produced the most conversions on the following landing page.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.