How a shift in copy tone affects clicks
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 12/23/2014 - 12/24/2014
In the fourth email in the Heritage year end campaign, we tested what messaging point (principles or family) would drive the most traffic to the landing page.
Research Question
An email talking about how Heritage impacts the constituents’ family life will drive more clicks than one talking about how they stand for the constituents’ principles.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Stand for your principles | 0.21% | ||
| T1: | Safer, happier, and more prosperous | 0.21% | -1.7% | 13.4% |
This experiment has a required sample size of 12,068,974 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
While the two treatments spoke to different prospect level value propositions, the copy did not seem to be different enough to see any significant impact on click throughs or donations.