Alliance Defending Freedom

How a shift in copy tone affects clicks

Experiment ID: #1344

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 12/23/2014 - 12/24/2014

In the fourth email in the Heritage year end campaign, we tested what messaging point (principles or family) would drive the most traffic to the landing page.

Research Question

An email talking about how Heritage impacts the constituents’ family life will drive more clicks than one talking about how they stand for the constituents’ principles.

Design

C: Stand for your principles
T1: Safer, happier, and more prosperous

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Stand for your principles 0.21%
T1: Safer, happier, and more prosperous 0.21%-1.7% 13.4%

This experiment has a required sample size of 12,068,974 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

While the two treatments spoke to different prospect level value propositions, the copy did not seem to be different enough to see any significant impact on click throughs or donations.


Experiment Documented by NextAfter

Question about experiment #1344

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.