How additional ad copy affects clickthrough rate
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 11/01/2015 - 11/30/2015
Dallas Theological Seminary was promoting their Christmas devotional series as a free offer on Facebook. Previous testing had shown that using copy in the image of the ad increased clickthrough rate and conversion rate. But as they reviewed their ad, they wondered if there was actually too much copy in the ad, and reading it bogged the reader down and kept them from clicking.
They made a treatment that removed the subheadline from their ad image, leaving only a single call-to-action with no supporting copy in order to simplify the value proposition.
They launched an A/B test to determine a winner.
Research Question
How does reducing copy in the image affect clickthrough rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 2.3% | ||
| T1: | Treatment 1 | 2.8% | 19.8% | 95.9% |
This experiment has a required sample size of 9,173 in order to be valid. Since the experiment had a total sample size of 19,763, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
19.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The ad with reduced copy produced a 19.8% increase in clickthrough rate, which indicated that less copy simplified the value proposition enough to attract more interest from people who saw it.