How “personalized” ad copy affects clickthrough rate
NextAfter
Experiment Summary
Timeframe: 09/17/2015 - 10/06/2015
The Texas State Historical Association launched a free eBook offer about Texas’ involvement in the Civil War. They wanted to test whether personalizing ad copy would increase clickthrough rate, so they made two variations of the same ad, with a slight but significant word change. The first set gave viewers the offer to get “the” free eBook, while the second personalized the offer with get “your” free eBook. We hypothesized that personalizing the offer would increase clickthrough rate and result in more email addresses acquired.
Research Question
Will personalized ad copy increase clickthrough rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 5.2% | ||
| T1: | Personalized copy | 3.9% | -24.6% | 100.0% |
This experiment has a required sample size of 2,180 in order to be valid. Since the experiment had a total sample size of 48,215, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
24.6% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment ad copy decreased conversion by nearly 25% — which showed us that viewers of the ads didn’t respond nearly as well to a personalized offer. This indicated that viewers of these ads didn’t feel that the book was already “theirs”, and that creating distance between the user and the offer enticed more of them to claim it. There are plenty of tests that show the opposite result, which means that it is always beneficial to test against “best practices”.