How authority, active language, and clarity affect email acquisition rate
Leadership Institute
Experiment Summary
Timeframe: 10/02/2017 - 11/09/2017
Leadership Institute recently re-launched a course about the history and principles of conservatism, called Conservatism 101. As they prepared to market this course through Facebook, they developed four different treatments for the advertisements. Testing proved that the third and fourth treatment produced a significant lift to clickthrough rate, but they wanted to gather more data to validate if one of these resulted in more downstream conversions—in this case, more sign-ups for the course.
The primary difference in the two treatments was the image and the copy within. The first treatment had a patriotic background that described what the course was: “Conservatism 101: An Introduction to the American Conservative Tradition”. The second, however, featured a picture of Mark Levin, a popular radio host and lecturer in the course. This image contained call-to-action copy as well: “Join Mark Levin in this new free course, Conservatism 101.”
The team isolated these two ads to determine which ad was more effective at driving downstream conversions.
Research Question
How does increased authority, active language, and clarity affect email acquisition rate?
Design
Results
This experiment has a required sample size of 6,454 in order to be valid. Since the experiment had a total sample size of 71,175, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
63.7% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The second treatment, with Mark Levin and the call-to-action copy in the image produced a 63.7% increase in downstream conversions. This showed that the increased authority and active call to action was able to shape motivation to affect downstream actions. Ultimately, Leadership Institute didn’t just want more clicks—they wanted more clicks with a higher motivation.
This test prompted a new round of testing that used the winner as the control to continue to increase results.