How adding specificity and urgency affects clickthrough rate
Compassion International
Experiment Summary
Timeframe: 10/22/2017 - 10/24/2017
Compassion International served Facebook ads to people who had browsed their site that showed them dynamically generated ads based on the children they had viewed. While results were positive for the campaign, they wanted to see if they could increase clickthrough rate and conversion rate by adding specificity to the ads.
They created a treatment that was identical to the first ad with one small change—in the call-to-action, instead of “Sponsor a child today”, they changed the copy to “Sponsor this child right now” to increase specificity and urgency.
They launched an experiment to determine a winner.
Research Question
Will increasing specificity and urgency in the call-to-action increase clickthrough rate and conversion rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.39% | ||
| T1: | Increased specificity and urgency | 1.3% | 233.4% | 100.0% |
This experiment has a required sample size of 395 in order to be valid. Since the experiment had a total sample size of 682,978, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
233.4% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment produced a 233.4% lift in clickthrough rate, but resulted in fewer downstream conversions (though this data was not valid). This suggests that while the ad might increase user motivation to click, something on the following landing page is not aligning with that motivation and is causing cognitive friction in the mind of the prospective sponsor. This shows that the experiment needs more data and possibly downstream conversion testing to discover what is causing the depression in conversion.